Diffusion language models (DLMs) sell parallelism: instead of emitting one token at a time, each forward pass predicts many positions at once, which makes them attractive for latency-critical generation. To escape the fixed canvas of early DLMs, most recent large-scale models — LLaDA 2.1, DiffusionGemma — use block diffusion: the sequence is split into blocks generated one after another, each refined with bidirectional attention while attending causally to finished blocks through a KV cache.

That design hides a structural cost: every block boundary is a synchronization barrier. The next block cannot be initialized until the current one is fully finalized, and a block is only as fast as its slowest token. Easy tokens settle early; a few stubborn ones keep self-editing; the whole pipeline waits.

The last 20% of a block yields 10% of the tokens

A new paper puts a number on this "boundary tax" ("Blurring the Block Boundaries: Overlapping Block Denoising for Efficient Diffusion Language Models", arXiv 2611.14387, by Amir Mann and six coauthors). For DiffusionGemma on HumanEval+, the final 20% of a block's progress consumes 22% of the forward passes but produces only 10% of its tokens. The token-share-to-step-share ratio in that tail falls to 0.35–0.71 on DiffusionGemma and 0.05–0.62 on LLaDA 2.1 mini: near the end of a block, most of every forward pass is wasted.

B3: two blocks share one forward pass

The proposed method, B3, is refreshingly direct. When the current block is "settled enough", the next block is spawned into the same forward pass and both are denoised together. Three design points matter:

  • Attention stays block-causal, so the current block's predictions stay identical to those of vanilla decoding, with no distribution drift.
  • Errors are absorbed by self-editing. The future block is drafted on a still-evolving prefix and may be misled early; once it becomes the current block, self-editing revises whatever it got wrong.
  • The spawn point calibrates itself. The gate opens when the maximum token entropy in the current block drops below a threshold τ — and τ is not tuned by hand but estimated from the model's own acceptance behavior during generation. No training, no hyperparameters, plug-and-play for any self-editing block DLM.

With B3 active, the tail ratio climbs back to 0.67–1.39 and 0.71–1.25: the idle tail of each block gets filled with real work.

Consistent gains across 30 model-benchmark pairs

The evaluation covers DiffusionGemma 26B, LLaDA 2.1 mini 16B and LLaDA 2.1 flash 100B — two diffusion families, 16B to 100B — on 10 benchmarks including GSM8K, MATH500, AIME26, GPQA-D and HumanEval+:

  • Average tokens-per-second (TPS) rises 11.5%–14.5%; tokens-per-forward (TPF) rises 15.9%–19.4%.
  • All 30 model-benchmark pairs improve, with no per-model or per-task tuning.
  • Average accuracy shifts only between −0.40 and +0.76 percentage points (+0.03 pp averaged over the three models).
  • Standout cells: DiffusionGemma 26B gains 20.3% TPS on AIME26 while adding +3.33 pp of accuracy; LLaDA 2.1 mini gains 21.9% TPS on HumanEval+.

Against two concurrent multi-block methods that rely on tuned thresholds, FlowBlock and MBD-LMs, B3 leads on both axes at block size 64: 11% faster than FlowBlock at comparable accuracy, and 3.0 pp more accurate than MBD-LMs at 3% higher TPS. Simply enlarging blocks does not buy the same thing either: at block size 256 B3 adds 12% TPS on DiffusionGemma, while the baseline needs blocks of 384 or larger to match that speed, losing at least 4.0 pp of quality.

They also fixed LLaDA's inference code

Two details deserve a note from anyone reproducing the work. First, the shipped Hugging Face implementation of LLaDA 2.1 iterates MoE experts with a plain Python for loop, driving GPU utilization "to near 0"; the authors replaced it with vLLM's fused MoE kernel. Second, without KV-caching of finalized blocks, runtimes for the 100B flash checkpoint grow quadratically with output length. Both fixes are orthogonal to B3 itself. Hardware: the 26B model runs on 2×A100 40GB, the 100B on 2×H200 141GB — all public checkpoints.

The limitations section is honest: B3 needs self-editing, so masked DLMs without an editing mechanism are out of scope; unlike verification-based speculative decoding it does not guarantee outputs identical to vanilla decoding; the speedup is capped by each block's low-acceptance tail; and the wall-clock numbers come from a single inference stack and English-only tasks.

The takeaway: this is scheduling-layer speed that any self-editing block DLM stack can pocket for free — no weight changes, no retraining, no knobs. And a closing footnote for the era we live in: the authors acknowledge in Appendix H that Claude Code helped write the code and LaTeX, with Claude Chat acting as a reviewer. Research on accelerating language models, itself accelerated by language models.

Paper: arXiv:2611.14387 · mirror: papers.cool.