Mechanistic interpretability is the effort to reverse-engineer what a neural network’s internal components — individual neurons, directions in its activation space, or larger circuits built from them — represent, and how those internal computations causally produce the network’s outputs. It is distinguished from behavioral evaluation, which only observes a model’s input-output pairs, by trying to open the “black box” itself: identifying interpretable features inside a trained network and tracing how they combine into the computation the network actually performs.
A recurring method is to find a direction in a model’s internal activation space that correlates with some human-interpretable concept, then intervene on that direction directly — amplifying or suppressing it during a forward pass — to test whether it plays a causal role in the model’s behavior, rather than merely correlating with it.
Related papers
- Sofroniew, Kauvar, Saunders, et al., “Emotion Concepts and their Function in a Large Language Model”, Transformer Circuits Thread / arXiv:2604.07729 (2026) — finds 171 internal emotion-concept directions in Claude Sonnet 4.5 and causally tests their effect on the model’s behavior. Covered in 2026-w32.