Diffusion language models (DLMs) do not decode left to right. They start from a fully masked block and iteratively unmask tokens over many denoising steps, which creates a quantization problem that simply does not exist in autoregressive models. A team from Technion and the INSIGHT Lab at Ben-Gurion University argues that the default inherited approach of uniform bit allocation wastes precision in precisely the wrong places. Their framework, T2Quant, treats quantization precision as a resource to be scheduled over the denoising timeline rather than a single global constant (paper: https://arxiv.org/abs/2610.31198).

Sensitivity depends on the task and on the moment

The paper's first finding is that activation quantization sensitivity in DLMs is not uniform. It varies along two axes at once, and the two axes turn out to matter independently.

Along the task axis, the three evaluated models produce three very different shapes. GSM8K is strongly front-loaded, with its peak inside the first 8 denoising steps. MMLU-Pro is also early but broad: its first four windows, covering steps 0 to 31, all sit within 10 percent of its peak. HumanEval behaves in the opposite way, barely sensitive early and peaking broadly across the middle of the trajectory, roughly steps 40 to 63. The project page puts the same finding in more concrete terms: along the W4A4 trajectory of LLaDA-8B-Instruct, GSM8K's very first window carries 27 percent of the task-relevant divergence, 4.4 times what uniform spreading would give, and the first quarter of the trajectory absorbs 55 percent. HumanEval is different again, with five mid-trajectory windows spanning steps 32 to 71 together accounting for 52 percent.

These profiles also turn out to be stable. Comparing the per-window curves of LLaDA-1.5 against LLaDA-8B-Instruct gives Pearson correlations of 0.99 on GSM8K, 0.98 on HumanEval and 0.92 on MMLU-Pro, meaning the temporal profile survives preference alignment.

A label-free signal for where precision pays

Methodologically, T2Quant deliberately avoids inventing a new quantizer. It does not touch the weights, the sampler, or the underlying quantizer. It only decides which bit width each denoising window runs at, drawing from the set {3, 4, 6, 8} bits while holding the average strictly at 4.00 bits.

The hard part is deciding which window deserves more precision without access to ground-truth labels or task-specific metrics. The paper solves this in two steps. First it generates a full-precision reference trajectory per calibration prompt and replays every saved state under each candidate width, isolating the local quantization effect and yielding a divergence measure for that window. Second it weights that divergence by downstream influence, defined as how much the tokens committed in that window actually move the model's beliefs about positions that are still masked. Multiplying the two gives an estimate of the marginal benefit of upgrading that window, and a dynamic program over windows and cumulative cost then selects the globally optimal allocation under the budget.

The influence term is not optional. Divergence alone misfires on HumanEval's final window, whose committed tokens cannot affect anything that follows.

The numbers

Evaluated on three masked-diffusion models, Dream-7B-Instruct, LLaDA-1.5-8B and LLaDA-8B-Instruct, across six benchmarks including GSM8K, HumanEval, ARC-Challenge, MMLU-Pro, DROP and BBH, with 4-bit weights and average activation precision fixed at 4.00 bits:

  • Average accuracy improves in all nine baseline-model combinations, with 42 of 54 benchmark-level comparisons improving. GSM8K and ARC-Challenge improve in every one of the nine combinations.
  • Gains over the corresponding baselines reach 6.7 percentage points on Dream, 3.4 on LLaDA-1.5 and 3.5 on LLaDA-8B-Instruct.
  • The best quantized configuration leads the strongest standalone baseline by a further 3.55, 1.13 and 1.40 points, leaving gaps to FP16 of only 2.0, 2.5 and 1.6 points.
  • On Dream, GSM8K accuracy climbs from 35.6 to 53.5 with DLLMQuant and from 46.1 to 58.8 with STaR-Quant. The other two models see GSM8K gains of 9.6 and 10.4 points.
  • The controls are the striking part. Random schedules under the same 4.00-bit average budget land 14 to 19 points below the original baselines on Dream, while RTN, which skips both rotation and GPTQ, collapses to 0.00.

The ablations point the same way. Sixteen unlabeled calibration prompts already recover about 75 percent of the gain. On LLaDA-8B-Instruct and GSM8K, uniform W4A4 scores 44.1, T2Quant scores 49.4 and FP16 scores 52.9. Squeeze the budget down to an average of 3.75 bits and T2Quant still reaches 48.4, beating uniform W4A4 at a full 4 bits. Pushing precision beyond 4 bits, by contrast, adds essentially nothing.

Where the caveats are

The paper flags its own limits. Hardware efficiency, meaning inference speed and energy consumption, is not measured at all, and all quantization is simulated in bf16 arithmetic. The precision schedule is task-dependent, so a new task means recalibration, and calibration needs representative unlabeled prompts from that task. Quantization itself can also alter safety-related behavior, which the authors list as future work.

For DLMs specifically, all three are separable. T2Quant is a scheduling layer that can sit on top of any post-training quantization method, binding to no particular quantizer and leaving the sampler alone.

The takeaway

The habit inherited from the autoregressive era is that precision is a global constant. DLMs break that premise. With the same weights and the same generation run, the importance of different denoising moments can differ by a factor of several. Turning precision from one number into a budget allocated over time is arguably the more transferable idea here, rather than just one more quantization trick.