Continual pre-training, also called mid-training, has a default pipeline: take a domain corpus, keep running next-token prediction on it. What this paper shows is that the same default recipe, on the same corpus, can push one model forward and drive another straight off a cliff.

An arXiv preprint submitted on 10 November 2026 (arXiv:2611.13855) documents this in a controlled experiment. The authors randomly sampled a 1B-token subset of MegaMath-Web-Pro, applied the same NTP mid-training recipe to two base models — Llama-3.2-3B and Qwen3-1.7B-Base — and got opposite results on MATH-500 and Minerva.

One recipe, one model rises, one collapses

Llama-3.2-3B started at 11.2 pass@16 on MATH-500 and 7.3 on Minerva, and after one round of mid-training rose to 37.0 and 17.3. That matches intuition: keep feeding mathematical text, mathematical ability goes up.

Qwen3-1.7B-Base went the other way, from 76.6 / 30.5 down to 18.2 / 5.5. Its MATH-500 avg@16 fell from 18.5 to 1.5, and its Minerva avg@16 from 4.7 to 0.4. The paper's NTP checkpoint table rules out mid-training noise: step 400 gives 30.2, step 800 gives 17.8, step 1200 gives 17.6, step 1600 gives 20.0, and the final step 1902 gives 18.2. There is no sign of recovery.

The authors' reading is that a stronger base has more developed reasoning behavior to preserve, and may therefore be more vulnerable to conflicting or low-utility training signals. Corpus quality, in other words, is not the only variable — the same corpus can have sharply different utility for different base models. This is exactly where the current mid-training pipeline is blind: mainstream practice leans on static data curation (heuristic filtering, deduplication, distribution matching, domain reweighting), all of which happens before optimization and none of which knows what states the model actually visits.

Swapping the objective: score where the model actually goes

The paper's proposal is called OPMD (On-Policy Mid-Training Distillation), and it recasts mid-training from offline text imitation into active, query-driven knowledge acquisition.

Three steps. First, anchor selection (Anchor Token Identification): a frozen lightweight proxy model, DeepSeek-R1-Distill-Qwen-1.5B, scans each document to compute per-token predictive entropy, with long documents processed in windows of 4096 and stride 2048, averaging entropy across overlapping regions. Three arms are then built — the top 5% highest-entropy positions, the bottom 5% lowest-entropy positions, and a stochastic arm preselecting each position independently with probability 0.05. Second, dedup and alignment: within a document, candidates are partitioned into non-overlapping 512-token buckets and only the highest-entropy candidate in each bucket is kept; the three sets are then uniformly downsampled to an identical 435,049 anchor prompts, so the comparison isolates where rollouts start rather than how much training happened.

Third, the core of it. The student takes the context of up to 1024 tokens preceding the anchor as a prompt and rolls out up to 512 tokens of continuation on its own; the frozen teacher then supplies dense token-level reverse-KL rewards on the states the student actually visits, restricted to the student's Top-16 next-token candidates. The implementation runs on the veRL PPO framework with a learning rate of 1×10^-6, a reference-policy KL coefficient of β=0.005, 4–8 NVIDIA A100/A800 GPUs, and one epoch of AdamW.

Is a 4B teacher enough

On results, with Qwen3-4B-Base as teacher, MATH-500 pass@16 reaches 82.8 (+6.2 over base) and avg@16 reaches 30.5 (+12.0); Minerva pass@16 reaches 34.9 (+4.4) and avg@16 7.0 (+2.3). Swap in Qwen3-14B-Base as teacher and MATH-500 gives 82.4 / 29.9 while Minerva gives 36.0 / 8.0.

The two are close to a tie, even though teacher capacity grew more than threefold. The paper offers two possible explanations: a capacity ceiling for the 1.7B student, since the soft-target signal over the Top-16 set is already sufficient; and state-space distributional alignment, because rollouts are generated by the student itself, so the visited states remain anchored inside the student's distribution and the two teachers assign similar relative log-probabilities to influential candidates there. On the two benchmarks tested, dropping teacher compute from 14B to 4B costs nothing meaningful — which is real money saved for anyone trying to reproduce this. The paper also cautions against comparing step counts directly: the 4B-teacher run took 1699 steps on 4 GPUs, the 14B-teacher run 1132 steps on 6 GPUs, and both correspond to equivalent overall token exposure.

One more detail worth keeping: convergence speed. In the 14B-teacher run, at step 100 MATH-500 avg@16 had already risen from 18.5 to 29.4, and Minerva from 4.7 to 7.1.

Where the anchor is picked matters less than you'd think

The ablation splits into high-entropy, low-entropy and stochastic arms, all held to the same 435,049-sample budget. The high-entropy arm delivers the highest peak avg@16 on both benchmarks — 31.9 on MATH-500 (step 800) and 8.1 on Minerva (step 800). The low-entropy arm is clearly worse, peaking at 25.3 (step 900) on MATH-500 and 6.6 (step 1699) on Minerva; the paper's explanation is that low-entropy positions are mostly highly deterministic structural or syntactic continuations, offering limited exploration value.

But the stochastic arm posts the higher peak pass@16: 83.2 on MATH-500 (step 800) and 40.8 on Minerva (step 400). So the OPMD paradigm itself is robust to where anchors land; converting static corpus context into active prompt-rollout triggers is the main contribution, and high-entropy selection's advantage shows up more in solution consistency (avg@16) than in peak score.

Three caveats worth stating plainly

First, the paper explicitly lists evaluation across additional domains, and with uniformly random final anchors, as future work. And the "stochastic" arm's final anchors are in fact not uniformly random — the dedup step still keeps the highest-entropy candidate in every 512-token bucket, and the authors say so in the appendix.

Second, the NTP and OPMD arms do not share a learning rate: NTP uses 1×10^-5. That matters when reading the "same corpus" framing. And the checkpoint figures in the paper's tables are successive evaluations from a single run, not independent repetitions.

There is also a reminder from an independent source. The evaluation scripts OPMD uses — gen_vllm.py and grade.py — come straight from the Rethinking On-Policy Distillation project by Tsinghua's THUNLP team (arXiv:2604.13016, accepted to TMLR). That paper argues OPD succeeds only under two conditions: student and teacher must share compatible thinking patterns, and even with consistent patterns and higher scores the teacher must offer genuinely new capabilities beyond what the student saw during training. It also finds that successful OPD is characterized by progressive alignment on high-probability tokens at student-visited states, with a small shared token set concentrating 97%–99% of the probability mass. More pointed: the authors show that OPD's apparent free lunch of dense token-level reward comes at a cost, raising the question of whether it can scale to long-horizon distillation. OPMD's teacher and student are both same-family Qwen3 bases, which sits in interesting tension with the observation that same-family teachers can be indistinguishable from the student's perspective.

So what

The value here isn't that yet another distillation method appeared. It's that a decision previously settled by intuition and trial-and-error — which objective function to use for mid-training — becomes an engineering variable you can validate cheaply. OPMD changes only the training procedure: no teacher at deployment, no extra inference overhead, and it drops into existing CPT pipelines.

The actionable version sounds like this: if your mid-training shows no visible gain, don't add more data yet — try a different objective function. One controlled comparison may well cost less than scraping a few million more tokens.