A small open-weight model answering a reasoning question has a default move: write a long chain of thought (CoT), then answer. A new paper from a Heriot-Watt University team in Edinburgh (https://arxiv.org/abs/2611.24598) re-does that trade-off: on 25 reasoning tasks, LLaMA 3.1 8B answers directly at a mean accuracy of 46 and with CoT at 69, at a cost of about 197 generated tokens per question; Qwen 2.5 7B sits at 52, 77 and 245 tokens. Rotate the last hidden state of the direct prompt towards one direction, once, before generation, and the mean scores reach 85 and 86 at 19 tokens per question.

Labelling the hidden states of the chain

The method has two steps. Step one: take each token's hidden state from the CoT run and patch it into the last position of the direct prompt for the same question. If the model then answers correctly, the state is labelled recovering; otherwise failed. On average only 17% of CoT states recover the answer on LLaMA 3.1 8B, and 28% on Qwen 2.5 7B. A linear probe separates the two classes: mean AUROC 0.84 (LLaMA, range 0.71 to 0.92) and 0.80 (Qwen, range 0.52 to 0.93). On Multistep Arithmetic, where the direct prompt answers correctly in 0.4% of cases, the probe still reaches 0.88. The probes also transfer across tasks: above chance on 587 of the 600 pairs of tasks, most strongly between tasks of the same reasoning type (0.71 versus 0.65 across types on LLaMA). The authors conclude that the model encodes the two classes differently.

Build a direction, rotate once

Step two: for each question, average its recovering states and its failed states, subtract half of the failed mean from the recovering mean, and normalise, giving the question's own task-solving direction. The last hidden state of the direct prompt is then rotated onto this direction once with SLERP, keeping its norm, at layer 29 for LLaMA and layer 25 for Qwen. The two means sit close together (about 27 degrees apart on LLaMA, 21 on Qwen), so the paper sweeps the weight on the failed mean: accuracy climbs as it rises to 0.5 and falls again at 0.75, moving away from the failed mean but also from the recovering mean. Results: direct answers rise from 46 to 85 on LLaMA 3.1 8B and from 52 to 86 on Qwen 2.5 7B, above their CoT scores of 69 and 77, a gain of 39 and 34 points over the direct prompt. The direction beats CoT on 24 of the 25 tasks for LLaMA and 19 for Qwen; three of Qwen's losses are tasks where its CoT already scores 90 to 97. Two controls show this is not a trick: rotating towards the failed mean drops accuracy to 9 and 16, and replacing the per-question direction with a task-level average leaves accuracy at the direct-prompt level (46 and 50). The direction is specific to each question.

Three counterintuitive points

First, on tasks where CoT hurts (Movie Recommendation and Ruin Names for both models), the direction still gains 22 to 36 points over the direct prompt. Second, length adapts to the task: where the answer can be read from the question, the model answers in 3 to 5 tokens; where it must be computed, the model writes short steps, 39 tokens on GSM8K against 179 under CoT for LLaMA (52 against 248 for Qwen), and 55 against 140 and 66 against 260 on Multistep Arithmetic. Third, on questions that both the CoT and direct prompts get wrong, the direction recovers the correct answer on 64% (LLaMA) and 70% (Qwen). On Qwen's weakest probing task (Causal Judgement, AUROC 0.52), the direction still lifts accuracy to 86. In the paper's Figure 1, a multistep arithmetic question answered directly as -40, with the CoT itself answering 80, yields the correct 48 after the rotation; on a GSM8K question answered directly as 121, the model writes '$21+$50=$71'.

The mechanism is not a one-off

The claim that models compute more internally than they say has independent support: Orgad et al. (https://arxiv.org/abs/2410.02707) find that internal representations encode far more information about output truthfulness than previously recognised, and Deng et al. (https://arxiv.org/abs/2405.14838) show that explicit CoT can be internalised step by step, so chains need not be generated. All scores in this article are the paper's own measurements; there is no third-party replication.

The cold water

Building a direction needs the question's own CoT and patch labels, so it cannot be applied to a new question as is: the authors tried a regressor that predicts the direction from the direct prompt alone, and accuracy stayed at the direct-prompt level. The study covers two instruction-tuned models of 7B and 8B, tasks with short, automatically checkable answers, and interventions at three late layers and one position. It is also a continuation of the same team's work: their previous paper (Findings of EMNLP 2026) ran the patching analysis only on GSM8K; this one extends it, and is not an independent replication. The code repository is public (https://github.com/hmehrafarin/hidden-state-directions).

The takeaway the authors put forward: by their measurements, CoT is not the shortest route to this accuracy, and the collected question-direction pairs can train a model to predict the direction directly. If that day comes, the CoT line on the inference bill could in theory shrink by about nine tenths. Is this distilling the chain into the model, or another elegant experiment that only holds at the 7B tier?