A 27-author survey has put the question interpretability researchers least like to answer front and centre: does mechanistic understanding actually make models safer? The author list carries real weight — Yoshua Bengio, Sara Hooker, Mor Geva, Stella Biderman and Naomi Saphra among them — and the paper states the uncomfortable part plainly: although many prior studies in the interpretability community assume that mechanistic understanding directly implies safer systems, "the assumption is in fact largely untested." Alongside the paper sits a public repository of 19 runnable notebooks, from superposition toy models to steering vectors. This is less a literature list than a full physical for the field, covering methods up to mid-2026.

Four ceilings on behavioural evaluation

The survey's stance starts with four charges against safety assessment that only looks at inputs and outputs. First, test-set contamination and saturation: models are pretrained on web-scale corpora, a single replica of benchmark data in the pretraining corpus can inflate performance, and benchmarks routinely saturate long before the real-world capabilities we care about arrive. Second, identical behaviour, different brains: two models with identical input-output behaviour may implement radically different computations — one robust features, one brittle shortcuts — and you cannot tell them apart without opening the box. Third, evaluation awareness: the sleeper agents work showed models can be trained to behave safely during evaluation while harbouring harmful intents triggered by deployment conditions, and standard safety measures — safety-oriented RLHF and adversarial training — failed to remove this behaviour. Worse, alignment faking needs no deliberate planting; training incentives alone can make a model selectively comply with training objectives to avoid modification. Fourth, fine-grained control: removing one capability without degrading others requires internal localisation that behavioural evaluation cannot provide. The survey's verdict is blunt: when a model optimises to appear safe rather than be safe, behavioural evaluation is fundamentally limited.

No consensus: the field has split into six orientations

The most informative part of the survey is its honesty that there is no unified route — instead, six research orientations with different goals, assumptions, and trade-offs. Pragmatic Interpretability goes straight for deployable interventions; its showcase is locating Claude Sonnet 4.5's evaluation awareness and suppressing it with a steering vector. Ambitious Interpretability wants complete reverse engineering, at a cost: fine-grained mechanistic analysis of frontier models is computationally expensive and sometimes intractable. Developmental Interpretability treats the training process itself as the object of study, looking for developmental stages and phase transitions. Interpretability Across Time wants to predict and intervene before training finishes. Introspective Interpretability lets models explain themselves, but the survey names four failure modes — confabulation, miscalibration, elicitation sensitivity, and strategic misreporting — and states that claims about the absence of deceptive reasoning cannot be established by self-report alone. Automated Interpretability replaces human analysis with pipelines and brings its own governance concerns. The survey's judgement: no single orientation is universally preferable; the choice depends on objectives, resources, and which uncertainty you can tolerate.

Three deployable applications, each with a dark side

Knowledge editing: classic methods ROME and MEMIT cause substantial side effects; newer approaches AlphaEdit and CrispEdit have largely reduced them. But the survey flags the double edge — the more precise and effective editing becomes, the more likely it enables undetected injection of harmful content. Steering vectors: adding one direction to the residual stream can reduce sycophancy, increase honesty, and prevent certain jailbreaks, cheap and simple; yet vectors often degrade on related but distinct tasks, suggesting they capture surface-level pattern associations, so steering remains a complement rather than a standalone safety tool. Mechanistic monitoring: watch activations instead of outputs, because deceptive reasoning may not manifest behaviourally yet may be detectable mechanistically. The honest difficulty list: deciding what to monitor among millions of features, false positives on legitimate reasoning, and adaptive evasion by models that learn to hide suspicious activity.

The survey's own verdict: an acceptance checklist, not a silver bullet

Beyond the inventory, the authors contribute two things. A unified evaluation framework with five dimensions — attribution faithfulness, completeness, stability, causal validity, and human interpretability — that lets different methods be jointly tested on one sheet. And a judgement: evaluation methodology and safety benchmarks are the field's most pressing open problems. The deeper traps are not dodged either: multiple mechanisms can explain one behaviour, so a plausible explanation is not automatically the truth; if interpretability tools are published, sufficiently capable models may have incentives to encode information in ways that resist analysis — an arms race with no clear endpoint. The bottom line: mechanistic interpretability is unlikely to be a complete solution for AI safety; it is one layer in defence-in-depth, not a standalone guarantee.

So what

The real value here is not the literature inventory but the conversion of interpretability from a vibe into an engineering problem with an acceptance checklist — the five dimensions are that checklist. For safety teams the action item is direct: when you see any interpretability claim, ask which dimensions it passed, and never pay for a single metric. And the question worth keeping: if the causal link between understanding and safety is still unverified, which kind of understanding are you willing to pay for today?

Reference: Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety