Video generation models improve year over year, and so does the inference bill. Nunchux AI's blog gives a concrete number: running MiniMax-H3 with BF16 FlashAttention-4 on a single B200, about two thirds of every denoising step is spent on attention; the longer the clip, the more that cost grows with the square of the token count. Attention is the choke point of video diffusion deployment cost.
Two Walls of Low-Bit Attention
Low-bit quantization is the obvious path to savings, but it hits two walls. The first is accuracy: a hardware block's quantization scale is set by its largest entries, so when outliers appear, typical entries get squeezed into a narrow representable range. Prior work already smooths queries and keys, but value outliers follow no fixed channel or spatiotemporal structure and remain the dominant source of output error. The second is speed: low-bit Tensor Cores accelerate only the two matrix multiplications; the softmax between them still runs in FP32, and that high-precision exponential becomes the longest pipeline stage of the kernel.
V-Smooth and ExpCast-FP8
VC-Attention tears down both walls. V-Smooth uses lightweight online k-means to regroup value tokens so that tokens in the same hardware block are similar; it then subtracts the block mean and quantizes only the residual, restoring the mean from the row sum that online softmax already maintains. Across four models, this step alone adds 1.1 to 2.8 dB of PSNR over SageAttention2. ExpCast-FP8 maps log-domain scores directly to E4M3 probability codes with one fused multiply-add, eliminating the FP32 exponential and the format conversion entirely.
Benchmark Numbers
The paper is implemented and evaluated on five GPUs — B200, B300, H200, RTX PRO 6000, and RTX 5090 — across four video models: Wan2.2, LongCat-Video, HunyuanVideo-1.5, and MiniMax-H3. Relative to BF16 FlashAttention-4, the attention kernel speeds up 1.46-1.59x on datacenter Blackwell and Hopper cards and 2.3-3.6x on workstation cards; end-to-end clip generation is 1.13-1.19x and 1.36-1.70x faster respectively. On the MiniMax-H3 workload generating 243 frames at 1344×768, the kernel is 1.59x faster on B200 and 1.51x on B300; fidelity across 100 prompts averages 20.2 dB PSNR, slightly ahead of SageAttention2's 19.9 dB.
Training-Free Is the Point
The paper comes from Nunchux AI, with authors including Song Han, Yujun Lin, and Lvmin Zhang. For engineers, the biggest selling point is not any single multiplier but the fact that it is training-free: no retraining, no weight changes — existing video models can swap in the attention kernel and take the speedup, and it composes with sparse attention, few-step distillation, and multi-GPU execution. As video generation moves from demo to production, every bit of inference cost gets multiplied millions of times over. The low-bit road cannot bypass softmax, and VC-Attention flattens a piece of that barrier.
So what? Model races are about parameters; cost wars are about kernels. Whoever moves attention into the 8-bit era first gets the more confident pricing sheet for their video API. The paper and the Nunchux blog post are both public — worth verifying the numbers yourself.