Streaming video generation has reached the few-step autoregressive generation: models can now generate and play back at the same time. But researchers from KAIST and NAVER AI Lab point out a waste hiding inside the inference loop: few-step autoregressive video generators run every single token through the full denoising schedule, regardless of how much refinement it actually needs. Uniform computation is the default, yet it is often unnecessary — many tokens already closely match their final outputs within the first steps, while others keep changing until the end.

Not every token deserves four full steps

The step-count-versus-quality ladder on LongLive (5-second videos) makes the point bluntly: 1 step scores 79.06 on VBench total, 2 steps 82.46, 3 steps 83.08, 4 steps 83.08. Going from 3 to 4 steps adds exactly zero to the total, while 1 step drops to 79.06 — so blindly cutting steps does not work either. The project page visualization goes further: background regions and areas similar to history converge early, while newly revealed regions keep needing denoising. Convergence is a per-token event, not a property of the whole clip — that is the starting point of the whole method.

Draft, verify, commit

Speculative Forcing splits into three stages. Draft: every denoising step already predicts a clean estimate for each token; treat that intermediate estimate as a draft of the token's final output. Verify: a verifier predicts how far each draft still is from the full-step output. The team built two designs — a training-free one based on similarity to clean content and stability across steps, and a learned one, a small 3D convolutional head reading the frozen generator's intermediate features. Commit: accepted tokens are immediately frozen as final outputs and skip all remaining denoising steps, while their cached keys and values stay available, so the rest of the tokens keep refining with full context. The generator's weights are never touched; at inference time a single tolerance ε trades speed against quality, with ε between 0.20 and 0.24 preserving four-step quality at the lowest Drift.

Three scenarios, three sets of numbers

  • 5-second short video (LongLive): the learned verifier averages 1.68 denoising steps per token, 41.74 fps on a single H100, VBench total 83.21 — versus the four-step baseline at 4.00 steps, 26.40 fps, 83.08. Quality edges up by 0.13 while throughput rises 1.58x; the training-free variant runs 1.65 steps at 43.61 fps with 83.12.
  • 30-second long video: 1.72 steps per token, 24.5 to 37.8 fps (1.54x), VBench-Long total 82.69 to 82.81, and Drift — the variation of imaging quality over the rollout — drops from 1.722 to 1.707. Freezing early does not hurt consistency; it actually steadies it.
  • World generation (Lingbot-World 2.0): camera control keeps revealing new content, so more tokens need the full schedule. The average climbs to 2.55 steps, yet throughput still goes 18.1 to 24.5 fps (1.35x), with Drift improving from 3.094 to 2.839.

Design details and the method's boundary

The ablation study carries two lessons. First, coarsening acceptance granularity from tokens to chunks commits whole chunks too early, and the errors propagate into later chunks. Second, using temporal difference as the acceptance signal cannot tell camera-induced changes from generation distortions. Together these failure modes show that the gains rest entirely on choosing the right granularity and the right signal. The 2.55-step world-generation result also draws the boundary: the more new content in the frame, the less computation there is to save.

In my view, this Draft-Verify-Commit decomposition is structurally the same bet as draft-then-verify acceleration in LLM inference: most positions do not need the most expensive computation. The difference is that "acceptance" in video is subtler — what gets frozen is the visual state of a spatial region, and the context must stay alive for later tokens.

For teams building streaming video generation and world-model rollouts, the practical meaning is direct: weight-free inference acceleration means deployed few-step generators can adopt it as-is, and the single ε knob dials the speed-quality point per scenario. As video generation converges toward real-time interaction, per-token compute allocation is becoming its own battlefield, much like LLM inference optimization. The code is open-sourced on GitHub.

Reference: arXiv:2610.22347 (arxiv.org/abs/2610.22347) | Project page speculative-forcing.github.io | Code github.com/naver-ai/speculative-forcing