Ask Qwen-2.5-72B for a 4×4 determinant and you will watch a nearly perfect performance: every product right, every sum right — and a final answer of 323 where the truth is 83. The mistake sits at exactly one position midway, a minus written where the algorithm demands a plus, and every step after it ran correctly on the corrupted expression. One character separates a perfect solution from a wrong one. A new 61-page paper dissects this failure class down to the layer level: arXiv:2611.07478.
Where the sign is written
The authors ran one per-layer readout pipeline across five open model families — Gemma-3-27B, Mistral-Small-24B, Phi-4-14B, Llama-3.3-70B-Instruct, and Qwen-2.5-72B-Instruct — on two deliberately small tasks: 4×4 determinants and integration by parts. The verdict is strikingly uniform: the wrong sign is committed by MLP layers in the last quarter of the network. On determinants, the share of confirmed errors peaking in that band is 72% for Gemma, 80% for Qwen, 94% for Llama, 95% for Mistral-Small, and 97% for Phi; integration by parts lands at 70–100%. Nor is this an instruction-tuning artifact: in Qwen, where tested, the base model already shows the pattern before tuning.
Stamps, amplifiers, and one rescue layer
Same location, different mechanisms. Qwen carries fixed "minus-stamp" layers (L75, L78) that vote minus regardless of the correct answer; a downstream rescue layer (L77) pushes hardest exactly where a plus should be written — Qwen errs when its standing minus default goes unopposed. Llama-3.3-70B is a pure amplifier: it magnifies whichever sign was already chosen. Column by column, the specificity test makes the split concrete. Qwen is the purest stamp: on determinants every one of its 11 error flips is corrective, none of its 104 plus-written cases is touched, and it wrote no plus at all in integration by parts. Gemma stamps errors but amplifies corrects, and is just as surgical — its minus cohorts collapse while 129 of 131 plus-written cases survive. Llama's lever moves minus-ness rather than wrongness: 43% of its plus-written determinant errors flip too, and 95–100% of its correct-minus answers break at the same dose. Mistral-Small is the outlier, breaking every cohort — surgical nowhere.
Localization does not imply steerability
The uncomfortable part is causal. Subtracting a single sign direction from the residual stream flips 13 of 14 held-out minus-written integration-by-parts errors on Llama-3.3-70B; the same intervention replicates on Qwen, Gemma, and Mistral-Small (24–70% per cell), and across the ten model×task cells steerability spans 4–93%. The controls are strict: a magnitude-matched random direction flips 1 of 923 pooled cases, and at the surgical dose none of Llama's 72 correct-plus answers breaks. Then there is Phi-4: equally well-localized at the same late depth, yet the identical intervention flips almost nothing — the sign is written redundantly across roughly 15 layers with a median logit margin of 17.8, several times any other model's. Margins and redundancy are invisible in a localization map; only intervening measures them.
Mechanistically, the paper names four properties that open the gap: redundancy, with several layers writing the sign along a shared direction so partial interventions are compensated by the rest; opposing structure, with Qwen's minus-stamps fought by rescue layers the steering never touches; stamps that carry minus-ness rather than wrongness; and sheer margin size. And the dissociation was itself on the record: among the frozen predictions was that the best-localized model would steer least — Phi-4's preregistered failure column came back exactly as predicted. Overall, 23 of 29 scored checks passed and the misses were reported. Code and per-case results are on GitHub.
The paper is also honest about scale. A sixth model, Mistral-Large-675B, was attempted but enters only as an exploratory behavioral run — the causal battery does not reach it. So the mechanistic core is five models between 14B and 70B, and the closest prior result (Hase et al.) showed only that causal-tracing localization fails to predict where weight editing succeeds; this work extends the dissociation to activation-level steering, and shows it is no accident of one model or one task — the same late decision, the same direction, and the same dissociation appear in both algebra and calculus.
So what
What the paper breaks is the default two-step program of mechanistic interpretability — find where a behavior lives, then intervene there. Finding the lesion does not mean the lesion is operable; margins and redundancy set the price of the knife. For anyone building model editing or white-box audits, a map is not a license to operate. For everyone else, there is a quieter comfort: when a model gets its math wrong, it often is not failing to reason — it mis-wrote one token. The next time a frontier model face-plants on arithmetic, the interesting question is not whether it can reason, but which layer stamped the wrong sign.