Running the same set of weights for more iterations at inference time is the cheapest way to work around "not enough parameters" — that is the entire motivation behind looped language models. But a paper posted to arXiv on October 20, 2026 pours cold water on it: more iterations do not mean you can handle longer problems.

Recurrent depth is not reasoning capacity; it collapses at 2x

The task setup is plain. The paper uses swap-S5: five boxes, one object in each, a sequence of swap operations, and finally a question about what is inside each box. Under the "single-query" setting, the authors first probe the limits of fixed-depth models: Qwen2.5 from 0.5B to 7B falls apart well past its training length. Trained on sequences of at most 32 moves and then fed 512-move sequences (16 times the training length), the 0.5B-to-7B range reaches only about 20% accuracy — roughly random guessing. The abstract records an even more extreme curve: explicit chain-of-thought (CoT) on the same backbone manages just 0.8% on swap-S5. Paper

In other words, the problem is not parameter count. Scaling to 7B still cannot keep up with the model's own training length; switching to having the model write out intermediate steps performs worse still.

Two root causes: supervision misalignment and state scale drift

The paper splits "running more iterations does not help" into two separable failures.

The first is supervision misalignment. When only the final answer supervises training, the model is never told "how far should iteration k advance" — nothing binds the number of iterations to task progress, so stacking rounds just sends the state drifting.

The second is scale drift. Applying the same weights hundreds of times pushes the scale of the recurrent state steadily out of distribution, and activation magnitudes unseen during training are enough to knock the model off track.

These map exactly onto the method's two changes, and the ablations assign credit cleanly.

PAR: make every iteration advance exactly one step

The method is almost anticlimactic in its simplicity. First, the prediction at iteration k targets "the true state after the first k operations" rather than the final answer — loop count and task progress are explicitly aligned, one loop per step. Second, after each loop the recurrent state is rescaled back to its first-iteration norm (norm anchoring), pinning the scale-drift variable down.

The result: Qwen2.5-0.5B trained on at most 32 moves reaches 95.5% mean full-state accuracy at 512 moves — exactly 16x length extrapolation. On the same backbone, the previously reported RLT method gets 55.7% and explicit chain-of-thought gets 0.8%. The authors add that Qwen2.5 at 0.5B, 1.5B, 3B, and 7B plus Qwen3 from 0.6B to 4B all steadily beat fixed-depth baselines, with the best Qwen2.5 training runs at four sizes hitting 100% accuracy at 16x the training length. The paper also topped Hugging Face's Daily Papers on the day, with 118 upvotes. Paper card

Ablations: neither component is optional

The ablations take "16x" apart cleanly: removing norm anchoring drops the extrapolation factor from 16x to below 2.2x — essentially nothing; replacing the progress-aligned targets with final-answer supervision caps it at 8x. State alignment and intermediate-prediction supervision have to ship together, and dropping either collapses the result.

One easily missed engineering detail: the model has to know when to stop. The paper uses a learned "completion readout" to decide when to emit the answer, without an externally supplied stopping depth.

What to do when intermediate labels are unavailable

swap-S5 naturally carries intermediate states, but real tasks usually offer only final answers. The paper offers a compromise: prefix self-distillation — inducing the "iteration k corresponds to step k" correspondence from shorter problem prefixes, using only final answers for training. The abstract's claim is that when intermediate labels are unavailable, this keeps the gains available.

The part actually worth watching

The weight of this paper is not the 95.5% figure but a repeatedly validated route under controlled experiments: train intermediate supervision and state scale together, and only then does recurrent depth become usable reasoning resource.

The hidden assumption behind "inference-time compute scaling" over the past two years is that giving a model more thinking time lets it handle longer inputs. PAR shows with controlled comparisons that this assumption does not hold on sequential state tracking — not because the model cannot think, but because nobody told it what to think at each round, or what its internal state should look like afterward.

What is interesting is that both gaps are training-side problems, not architectural ones. Loop models add no parameters at all; the paper changes only the supervision signal and one rescaling step. For teams currently piling on inference budget, that is a reminder: before spending budget on repeated calls at inference time, confirm each iteration carries an explicit progress semantics.