Diffusion language models have a well-known asymmetry: a single forward pass can predict many tokens at once, yet the model needs many decoding rounds before a response actually finishes. The parallelism is there; the round count is what kills latency. A paper submitted to arXiv on October 26 by Aidar Myrzakhan and Zhiqiang Shen of VILA Lab, MBZUAI, argues that the bottleneck is not which tokens are easy to predict, but which one you reveal first. The proposed framework, dSpark, treats the reveal order itself as knowledge that can be learned and distilled.
The counterintuitive finding: the most confident token is often the least worth revealing first
Confidence-based decoders commit the tokens the model is most sure about and defer the uncertain ones. The measurements here point the other way: on correct GSM8K responses from LLaDA, revealing an "uncertain" token — the kind confidence-based decoders would defer — lowers the surprisal of the other masked positions 4.7x more and makes 5.1x more of them ready, compared with revealing a confident one. The most predictable token often provides the least context; reveal the hard piece first and the puzzle assembles faster. The paper also observes that responses with a larger uncertainty area need more forward passes — round cost hangs directly on how much information is still unrevealed.
Method: a frozen teacher ranks a new reveal order, then distills it into the student
dSpark works in two steps. First, a frozen teacher re-reveals the same correct response in a new order: each round it promotes the token that most reduces the uncertainty of the remaining masked positions. Second, those promoted trajectories are distilled into the student so the model learns the schedule itself, with a KL anchor on the model's own failed responses to keep fast decoding from wrecking quality — in the README's phrasing, correct trajectories teach the schedule while failed ones serve as the regularizer.
Training data is fully self-generated: the base model produces responses on GSM8K, PRM12K, NuminaMath-CoT and AceCode (plus OpenMathInstruct-2 for the Dream base), with at most 256 new tokens and a block size of 32; responses are checked against reference answers or unit tests and split into correct and failed trajectories for schedule learning and KL regularization respectively.
Numbers and caveats
Per the paper's own reports, dSpark cuts forward passes on GSM8K by 10.12x on LLaDA-8B-Instruct (10.12 tokens per forward) and by 5.12x on Dream-7B-Instruct, while matching or improving accuracy. Evaluation covers GSM8K, MATH, HumanEval and MBPP via lm-evaluation-harness, reporting accuracy plus tokens-per-forward decoding speed. These are author-reported numbers with no third-party replication yet.
The cold water: the repo states that model checkpoints and training data "will be released soon" — at the time of writing they are not out; experiments concentrate on math and code tasks with two open 8B-class base models; the training config caps responses at 256 new tokens, so long-form generation is untested.
So what
The dLLM speed race is switching axes. The last phase competed on predicting better — converging in fewer rounds. This one competes on scheduling smarter: same model, change only "who gets revealed first", and forward passes differ by an order of magnitude (paper-reported). For inference-engine builders that is worth more attention than another benchmark point — scheduling is a zero-parameter acceleration lever, and one that can be distilled and transferred. Until the weights ship and someone replicates on fresh tasks, read "10x" as the paper's own claim. Paper / Code