Generating a single 480x832, 81-frame video with Wan2.1-I2V-14B takes 851.5 seconds end to end on one A100 — more than 14 minutes. The Diffusion Transformer denoises roughly 32.8k latent tokens over 50 steps, and that is where the compute bill comes from. A joint team from KAIST AI and Kakao proposes a different way to save: leave the pretrained diffusion model alone, and recompress its fuel — the latent space. The method, GRACE (Generation-Aware Latent Compression for Efficient Video Generation), is on arXiv (2610.10524), with code released under MIT and weights on Hugging Face.
Why compressing latents is hard
The speed bottleneck of video diffusion is token count: the higher the autoencoder's compression ratio, the fewer tokens the DiT processes, and the faster generation gets. But push compression too far and reconstruction quality collapses first — recovering detail requires more channels, which slows DiT convergence. Worse, the compressed latent distribution drifts away from what the DiT learned during pretraining, so the DiT must be retrained from scratch or adapted at considerable cost. Optimizing the encoder for reconstruction alone looks compatible on paper, but silently moves the latent space out of the world the DiT knows.
The three pieces of GRACE
GRACE works in two stages. Stage 1 trains the autoencoder: a frozen base latent from the original Wan2.1 encoder anchors the representation, while a learned residual latent carries the detail and motion that stronger compression throws away. Beyond reconstruction losses, the compressed latent is aligned with the pretrained one inside the frozen DiT's feature space, so the encoder is optimized for generation rather than reconstruction alone. Stage 2 adapts the DiT: input and output projections are extended to the residual channels, LoRA is attached to the transformer blocks, and asymmetric denoising keeps the base latent denoised slightly ahead of the residual (default offset delta = 0.15), so the residual always builds on a more settled base, at no extra sampling cost. The latent configuration goes from f8t4p2 to f16t8p2, and tokens at 480x832 drop from 32.8k to 4.3k.
11x faster, and T2V scores go up
The results table is the persuasive part: at 480x832x81, GRACE takes 75.8 seconds end to end versus 851.5 seconds for the original Wan2.1-14B — an 11.1x speedup — and at 736x1280x81 the speedup grows to 15.5x. On quality, GRACE scores 85.81 on VBench text-to-video total, beating the uncompressed original's 83.93, with the semantic score gap the widest (84.98 vs 78.70); image-to-video total is 87.90 versus 87.92 for the original, essentially flat. Among methods at the same token budget — Open-Sora 2.0, DC-Gen, LTX-Video — none matches GRACE's T2V total, while LTX-Video's I2V total (87.97) sits in the same tier.
Ablations: never waste the pretraining
The ablation study makes clear why brute force fails: with the same compression, training the DiT from scratch for the same number of steps yields a VBench T2V total of just 71.04 — 11 points below the 82.12 of the compress-only baseline. Adding the dual-latent design (83.55), the DiT-feature-space alignment (84.68), and asymmetric denoising (85.81) climbs the score back above the original. The alignment target matters too: video foundation model V-JEPA 2.1 features reach only 83.99, below the frozen DiT's own features at 84.68 — in generation tasks, the DiT's own worldview is worth more. Denoising order is directional as well: base-first scores 87.86 on I2V total, while residual-first drops to 87.17, below even the synchronous schedule's 87.51.
The significance goes beyond a single number: the compute bill of a pretrained video diffusion model can be renegotiated at the latent layer — no DiT retraining, an order of magnitude fewer tokens, and quality holds. As video generation costs stay high, generation-aware compression may well become a standard step in the pipeline. Code and weights are out, reproduction cost is low, and it is worth trying hands-on.
Reference: arXiv:2610.10524 / cvlab-kaist.github.io/GRACE