In reasoning-model circles, "wait" has long been credited to reinforcement learning. A model writes "Wait, is that correct?" before re-checking a step, and the assumption was that RL taught it that habit — the visible sign of reasoning being acquired. Researchers from HKUST, Huawei Noah's Ark Lab and Peking University traced 120 training endpoints of open models and reached a different conclusion. Paper

42.9 points of accuracy, and almost no "wait"

The cleanest control is zero-RL: the model trains only on its own sampled solutions, with verifiable rewards applied to the final answer. The team ran 18 such runs starting from 9 bases spanning Qwen2.5, Mistral and Qwen3 (0.5B to 32B) that had no marker to begin with. Accuracy on MATH-500 rose by a median of 42.9 points. And in 17 of those 18 runs the marker stayed below 0.05 occurrences per 1k tokens, with the exact 95% upper bound never crossing 0.06.

The pattern survives beyond a single benchmark. On AIME24, AIME25, AMC23, GSM8K and MMLU-Pro, zero-RL from Qwen2.5-7B never writes the word more than 3 times. By contrast, OLMo-3 RL-Zero raises it on every benchmark, from 4.62 to 17.10 per 1k on GSM8K, and only under MMLU-Pro's official few-shot prompt does it barely register at all (0.03 to 0.22 per 1k).

The marker arrives with the data

So where does it come from? Training data. The threshold for "does not have it" is 0.05 occurrences per 1k tokens. In OLMo-3 the word is essentially absent after pretraining (at most 0.04 per 1k), then rises to 4.03 after mid-training, with 88.6% of occurrences opening their sentence — right at the point where thinking traces were folded in for RL-Zero. Bases whose pretraining mixture already contains reasoning traces write it too: MiMo-7B-Base at 4.95 per 1k and SmolLM3-3B-Base at 4.18.

Fine-tuning reproduces the same thing. Of 14 models fine-tuned on reasoning traces from marker-free bases, 13 end up writing the marker at 0.5 per 1k or more. Marker forms track whatever the traces contained: OpenReasoning-Nemotron-7B opens sentences with "but" (98.3 per 1k) and "perhaps" (28.1 per 1k) instead of "wait" (0.2 per 1k).

What RL actually does: multiply, and wear down

Where the base already uses the marker, both released RL-Zero recipes multiply it. Along OLMo-3 RL-Zero the rate rises from 4.9 to a peak of 17.0 per 1k at step 1900, ending at 12.4 at the released checkpoint (12.68 ± 1.55 over 6 decoding seeds). MiMo-7B goes from 4.95 to 6.62. A KL-free GRPO run from the same base does not. RL after supervised fine-tuning also amplifies it in smaller steps: Phi-4 goes 1.56 to 2.58, DeepScaleR 3.65 to 4.07, Light-R1 2.79 to 3.21.

The form erodes at the same time. Position stays put — 89.6% of occurrences open their sentence at the base and 89.4% do at the end — but the canonical "Wait," share falls from 88.9% to 37.5% as the comma drops. Reduplicated chains ("Wait, wait") rise from 117.6 to 213.0 per 1k occurrences, and marker-initial sentences shrink from 11.8 to 6.1 words. The paper reads this through grammaticalization: a late-stage marker loses substance and binds to fixed position. Usage drifts too, with a Jensen-Shannon divergence of 0.05 between the last checkpoint and the base, against a noise floor of 0.002.

Does the word still mean doubt?

Multiplying the word did not automatically change what it means. The team built a test that needs no labels: take a prefix of correct steps, and a parallel copy with one injected arithmetic error, then check whether a revision marker appears within two sentences. The measure is d′, how well the marker separates errors from correct steps.

Along RL-Zero the marker becomes both more sensitive and more liberal: d′ rises from 0.57 to 1.09 (95% CI [0.97, 1.24]), but the criterion falls from 0.11 to −0.24. Taken alone, "wait" sees d′ rise from 0.77 to 1.29 (CI [1.16, 1.44]) while c′ falls from 0.46 to 0.01. It follows 73.7% of error prefixes, up from 47.1% at the base — but also 25.4% of error-free ones, up from 19.9%. After SFT, RL leaves the alarm almost unchanged: the criterion moves from −0.14 to −0.19 at 7B, and d′ from 0.82 to 1.02.

The sharper warning: a judge reversed the trend

The paper also ran a 60-sentence blind audit with expert consensus labels on sentences drawn from the base and the final RL-Zero checkpoint. The primary LLM judge agrees with consensus at κ = 0.64 — a respectable number. Yet on the same weighted items its revision rate falls from 26.0% to 12.9%, while the human consensus rate rises from 1.3% to 15.9%. It gets the direction backwards. A second judge tracks consensus correctly (1.5% to 12.9%, κ = 0.71).

So what

This reads as a story about a measuring instrument used carelessly. Counting "wait" and "hmm" in training logs has become standard evidence that reflection is increasing, and an LLM judge is often bolted on to label the behaviour. The evidence chain here says otherwise: whether the marker appears at all depends on whether the training data already contained it; RL mostly multiplies and wears down inherited material rather than inventing it; and the marker's evidential weight shifts with the recipe and the checkpoint.

So the next time a model starts "reflecting" a lot, the first question should perhaps not be "what did RL teach it" but "was this already in the data". The authors are careful about how far the finding travels: the evidence is observational, the zero-RL result rests on 18 runs over 0.5B–32B models, English mathematics and outcome rewards, and frontier-scale RL, process rewards and other languages all remain untested. Still, for anyone building reasoning models, the actionable part is clear — measure the marker directly on the checkpoint you care about, and never let an automated judge stand in for a human consensus without validating it.