Autoregressive LLMs have one built-in bottleneck: the next token depends on the previous one, so decoding is strictly sequential and GPUs spend much of their time waiting. Speculative decoding attacks this with a small draft model that guesses ahead, but the draft model itself needs training and memory, and its speedup collapses at large batch sizes. A paper posted to arXiv on September 3, "Unlocking Lossless Speedups in LLMs via Discrete Diffusion" (arXiv:2609.04010), proposes a third path: keep the AR model intact and bolt on a set of lightweight diffusion weights that draw multiple tokens in parallel. The work comes from the Institute of Foundation Models (IFM), with collaborators from Cerebras Systems and several universities.
Two weight sets, separate jobs
The paper introduces diffusion-augmented LLMs, a class of models with two parameter sets: AR weights trained with the standard next-token-prediction objective, responsible for quality, and lightweight diffusion weights trained to generate multiple tokens simultaneously. The latter are learned through a Diffusion Distillation phase that the authors say adds negligible overhead to existing LLM training pipelines. A family of samplers called Ψ-Spec draws tokens in parallel from the AR-defined distribution, enabling lossless acceleration and inference-time scaling at a fixed context length. Versus speculative decoding, the biggest difference is that no separate draft model is required; versus diffusion LLMs (d-LLMs), the quality of the underlying AR model is not sacrificed. The project page comparison also shows Uno introduces the least additional parameters and the lowest inference-time GPU memory among the evaluated methods.
An 8B model beating a 26B d-LLM
Two result lines stand out. First, Uno achieves higher throughput than leading speculative-decoding methods — the paper names DFlash and Eagle3 — at every evaluated batch size, with up to 3x speedups over the base AR model, including at the largest batch size supported by the device; the project page describes this as Pareto-dominating speculative decoding. Second, the authors report that the 8B Uno model outperforms the 26B DiffusionGemma, which the paper calls the leading open d-LLM, and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning. These numbers are self-reported and await community replication, but the direction matters: speedup without trading away quality.
So what
Uno can be trained from scratch or built by augmenting existing open-weight AR LLMs, and code plus checkpoints are already open (github.com/ifm-ai/uno). The paper also sat at the top of HuggingFace Daily Papers for September 8. For inference infra teams the lesson is that diffusion does not have to replace autoregression — it can be attached to an existing stack as a parallel-decoding bolt-on. If you run speculative decoding in production, this draft-free route is worth a benchmark pass. Full paper: arxiv.org/abs/2609.04010.