Here is an uncomfortable thought: a 64-layer Transformer, before writing a single token, may already have 'decided' what its next step is - and that decision can be read out of its internals and even overwritten.

This is not hand-waving. A new arXiv preprint, Goal Encoding in LLMs (https://arxiv.org/abs/2611.17297), takes the philosophical question of whether language models have goals and breaks it into three things you can measure in a lab.

Defining what counts as a goal

The authors borrow Bratman's planning theory and define a goal as a functional state with three required properties. Pre-commitment: the target state is settled before the action that realizes it begins. Plasticity: the same goal can be realized by different means, depending on what the environment demands. Persistence: the encoding is sustained while the action is carried out, and released once it is done.

All three sound natural for a person. All three are hard for a language model. A Transformer has no recurrence and no register, so there is no obvious place to park a thought like 'I am currently working on X'. The whole paper is an attempt to find such a place.

122 heads that know the next step

The setting is multi-step coding: the prompt lists sub-tasks - say, implementing merge sort - and the model writes them one at a time. At the position where the model is about to begin the next sub-task, the authors search for attention heads whose outputs carry an abstract representation of that sub-goal. Abstract matters: a representation tied to the exact tokens to be written would not be plastic.

On Qwen-2.5-Coder-32B-Instruct (64 layers), the search settles on 122 attention heads, spread across layers 28 to 55 and concentrated in layers 40 to 52 - 4.8% of all heads. Three results follow.

Pre-commitment: at the end of the previous line, the readout is already legible roughly 60% of the time. A few tokens later, at the indentation point, the readable rate exceeds 0.90. The model knows what comes next several steps before it starts writing it.

Plasticity: move these heads' outputs into a C++ context and, by the paper's own measurement, 96% of the time the model triggers the same sub-goal through a different sequence of tokens (about 87% on average when starting from the indentation point). The encoding defines what to do, not how to do it.

Causal bottleneck: cut off the access of roughly 93% of all other heads above layer 20 to the task description, and the model still writes the next sub-goal using only those 122 heads. Cut off just those 122 instead and performance barely drops; it degrades sharply only when a broader set of 250 heads is blinded. There is redundancy inside.

The probe is called GoalScope: take the selected heads' output vector, inject it into a separate run of the model, and let the model say in plain language what it is about to do next.

Not enough layers, so read it again

Here is the hard contradiction. A Transformer can only move state forward through an attention read, and each read costs a layer. The goal encoding is only readable after layer 40, which leaves 64 minus 40 equals 24 tokens of headroom - yet in the experiments the encoding is still there more than 30 tokens later. Mechanically, that should not be possible.

The explanation the authors propose is NRGS, a non-recurrent goal-seeking algorithm. The model does not carry the goal forward. It repeatedly re-reads the goal from the task description in the prompt, re-encodes the progress so far from the growing context, compares the two, and pursues whatever remains.

The intervention experiments support this. Transplanting the key-value states of the task-description span from a source prompt into the destination prompt makes the model implement the source prompt's next sub-goal in about 94% of cases, and the readout shifts accordingly about 90% of the time; transplanting the code-progress span instead is far weaker - only 6% - while transplanting the task span alone gives 89%, and both spans together reach 98%. Query transplants are weaker still: roughly 57% when the target sub-goal is absent from the destination list, rising to 87% when positions align, against about 3% for the same number of random heads.

The authors are careful here, calling this encouraging but incomplete evidence. They also note that when the source prompt's next sub-goal is present in the destination's task list anyway, the query transplant fires it about 96% of the time - suggesting the choice was already made in earlier layers.

Does it generalize beyond code?

The heads were localized in pure code generation, but the readout was tested elsewhere.

In few-shot in-context learning, reading out the implicit rule succeeds over 80% of the time for relations such as antonym, country-capital and person-occupation, but drops below 20% for translation tasks such as English-to-Spanish. Averaged over all 39 single-relation tasks the accuracy is 36.2%±5.6%, with 49.9%±6.4% on abstractive tasks and just 9.0%±5.7% on extractive ones.

As a control, the authors localize function vector heads using the public code and data from Todd et al.'s ICLR 2024 work. Those heads barely read out the rule at all: 2.3% on abstractive tasks and 1.5% across all 39 tasks, against 36.2% for the 122 goal heads. Two sets of heads, clearly different jobs.

On a misaligned model organism - Qwen2.5-32B-Instruct fine-tuned to give risky financial advice - the goal heads read out an intent to advise drastic action, while the innocent model reads out benign advice. That one is qualitative.

The appendix repeats the NRGS experiments on 4 other models from different families and sizes, with the pattern holding up to 106B parameters (GLM-4.5-Air, a MoE with 12B active parameters).

An independent team pushes back

Planning ahead is not a one-lab story. Anthropic's circuit tracing work on Claude 3.5 Haiku found a similar structure: before writing a line of verse the model activates candidate rhyme words internally, and inhibiting the preferred one (rabbit) makes the model rewrite the line to end on another (habit) (https://transformer-circuits.pub/2025/attribution-graphs/biology.html).

But a separate team, using lighter probes across Qwen3, Gemma-3 and Llama-3 at more than ten scales, reached a less comfortable conclusion: only Gemma-3-27B causally relies on that encoding, while every other model shows strong probe signal with near-zero causal effect (https://arxiv.org/abs/2605.07984). Being able to read something out is not the same as the model using it.

The boundaries the paper admits

The findings hold in a narrow setting: simple coding tasks, sub-tasks written out in the prompt, no chain of thought. Whether a goal survives an obstacle planted mid-execution is inconsistent at this scale and was left out of the main claims. Larger models may be less dependent on re-reading, and architectures with linear attention may retain a running state instead.

The most important line sits in the limitations: in more open-ended agentic settings the model may diverge from what the prompt specifies, and the NRGS account probably cannot explain that.

So what

The value of this paper is not the conclusion that LLMs have goals - the authors themselves are unsure whether this qualifies in Bratman's sense. The value is that 'goal' stops being a philosophical adjective and becomes something you can locate, read out, overwrite and ablate inside a model.

For anyone building agents, the practical implication runs the other way: if you want a model not to drift, do not count on it remembering the task. Make sure it can keep re-reading the task. NRGS sounds exotic, but the engineering version is simple - put the constraint in the context and give the model a reason to read it again at every step.