Edd Gent in Singularity Hub:
The idea that the neural networks underpinning modern AI are “black boxes” almost impossible to understand has become received wisdom. But a provocative blog post from a researcher who played a key role in setting that narrative says these models may be far more interpretable than we thought.
The opacity of today’s leading AI models is becoming an increasingly pressing problem as the technology seeps into ever more areas of our lives. Recent incidents where agents from leading developers escaped their sandboxes to go on hacking sprees have only boosted the sense of urgency. This has spurred interest in a research field known as mechanistic interpretability, which attempts to reverse engineer neural networks to learn what individual neurons and the circuits they create do. By detailing how these networks operate, researchers hope it’ll be easier to control and direct them.
More here.
Enjoying the content on 3QD? Help keep us going by donating now.