Sudoku is a clean testbed for self-correction: every cell has exactly one correct digit, so right and wrong are unambiguous. The authors decoded 4x4 Sudoku puzzles with LLaDA-8B-Instruct after one epoch of supervised training on Sudoku propagation traces, then ran a brutal experiment: on 77 failed boards they re-masked every wrong cell and let the model refill them. Only 6 boards recovered — even with the exact error locations handed over, the repair rate stayed under one in ten. The stranger part is that the error is plainly visible inside the model: a linear probe on final-layer hidden states separates wrong cells from correct ones at AUROC 0.838, while token confidence manages only 0.685. Errors can be found but not fixed in place — that is the real situation for masked diffusion language models (MDLMs), and the starting point of the new paper arXiv:2611.17320.

Why in-place repair fails

Both diagnostic findings point at the context defending the mistake. First, when a single committed wrong cell is masked again for refill, the original wrong digit keeps a mean probability of 0.869; re-opened sixteen steps after its commit, the probability rises to 0.962 — the unchanged surrounding cells keep supporting the same error. Second, rolling back the reasoning written after the error as well, the best fixed rollback depth repairs only 13 boards; even the best per-puzzle depth chosen with the gold answer reaches 15. The authors' verdict is blunt: error information is readable from the model's states, but re-predicting erroneous positions under their context rarely repairs them. The right move is to regenerate a region jointly and judge the alternatives by their eventual outcome.

DiVR: a verifier inside the generator

Hence DiVR (Diffusion Verifier for Regeneration). It reuses the generator's frozen backbone with a rank-32 LoRA adapter and a two-layer MLP head; trainable parameters are about 1% of the backbone. The training signal is only final-answer correctness: four sampled trajectories per question (temperature 0.7) over 2,000 GSM8K training questions, with no token-level labels. During decoding, whenever the generator finishes a 32-token block, the verifier scores the canvas; if the score falls below the 0.5 threshold and the budget of three regenerations per problem is not exhausted, the whole block is re-masked, four alternatives are sampled, and the best-scoring one is kept — otherwise the original stays. Canvases hold 256 tokens (512 for MATH-500), with 16 steps per block. The generator itself is never touched.

The ledger, as measured in the paper

Across five benchmarks (GSM8K, MATH-500, Countdown, HumanEval, MBPP), the fine-tuned LLaDA-8B-Base reaches 35.3% average with greedy decoding and 43.4% with DiVR-guided decoding at 1.9–3.5x the cost; in the same table the search baseline SR-SMC gets 39.1% at 3.8–4.0x and Prism 37.3% at 5.2–7.4x. Guided decoding thus uses about 70% of SR-SMC's compute and less than half of Prism's. On GSM8K alone, going from 68.7% to 78.8% costs 2.1x, against SR-SMC's 72.6% at 4.0x and Prism's 71.9% at 5.9x. It carries across generators: LLaDA-8B-Instruct goes from 39.4% to 44.8% (best-of-4 mode: 49.8%); Dream-v0-Instruct-7B from 45.0% to 54.8%, with Countdown at 18.0% to 37.2%, which the paper describes as a doubling. Generalization holds too: a verifier trained only on GSM8K lifts MATH-500 from 24.6% to 33.8%; the SFT generator's verifiers applied to the untuned Base raise its average from 29.3% to 38.0%, above the 35.3% of fine-tuning itself. DiVR also stacks with post-training: after diffu-GRPO reinforcement learning the average is 40.1%, and guidance takes it to 45.4%. The paper records an instructive detail: RL pushed the reward's correctness term from 0.76 to 1.35 out of a possible 2 while greedy accuracy barely moved — RL changes what the generator writes, the verifier decides which block to discard, and the two gains add.

The judge matters, not the sampling

Holding the regeneration budget fixed and swapping only the signal: commit-confidence gets 72.2%, prompted self Yes/No 67.9%, a head-only verifier 74.2%, and full DiVR 78.8% at ×2.10, below the 2.45–3.62 of the control signals. When selecting among four finished answers, mean log-probability reaches 68.5%, nearly the same as picking at random (68.7%); majority vote gets 75.7%, DiVR 82.2%, with the oracle ceiling at 85.4%. Repairs cluster at the start: of GSM8K's 154 repaired problems, 74% depart at block 0 and 92% within two blocks; the block-0 score flags 76% of eventually wrong solutions with 22% false positives, and Countdown sits at 99% versus 1%. Sweeping the threshold from 0.1 to 0.9 moves accuracy only within 77.5–79.2% while cost runs 1.79–2.61x — the threshold is mainly a cost knob, not an effect switch.

The caveats

The paper's own limits deserve quoting: the primary run hits 78.8%, changing only the verifier-training seed gives 77.5% and changing only the re-decoding seed 77.6%, which the authors call sensitivity checks rather than independent end-to-end replications; outcome supervision depends on answer-checkable domains — math and code qualify naturally, open-ended tasks need something else. The conclusion can be paraphrased as: what revision in MDLMs lacks is a learned criterion for when to start over, more than a finer way to find the error. That continues the autoregressive tradition of training verifiers on final-answer correctness and using them to pick candidates; DiVR moves it onto the partially masked canvas. The code repository is linked in the paper footnote: minhaeoh/DiVR.

The model knows it is wrong; it just cannot fix itself. Letting a judge that shares the backbone and trains about 1% of its parameters call for a redo is cheaper and more effective than letting the model stare at its own confidence — for anyone doing diffusion-LM inference, that is the takeaway to steal.