Autoregressive (AR) video diffusion models are the mainstream path for long video generation today, but the pain point is well known: as generation time stretches, the characters and objects in the frame gradually "change faces," and details start to collapse — the industry calls this identity drift. On June 1, NVIDIA Lab (NVlabs) released LongLive-RAG on arXiv, bringing the RAG idea systematically into long video generation for the first time.

Why isn't sliding window enough? Existing methods generally adopt sliding window attention to control VRAM, but this mechanism has an irreversible trajectory bias: once the current window accumulates appearance errors, subsequent generation can only continue forward based on this "damaged" trajectory, drifting further and further.

LongLive-RAG's core solution: treat already-generated latents as retrievable memory. Each new block retrieves the most relevant historical latents via query embedding to participate in conditional computation, letting the generator "look back" at non-local context rather than just staring at the most recent few frames.

Supporting Window Temporal Delta Loss suppresses the retriever's preference for redundant local similarity, encouraging the embedding to capture meaningful temporal change — this avoids the degradation of "retriever only picks up the just-generated frame."

Overhead is tiny: retrieval per block adds only 4.08ms, total retrieval overhead 490ms. Experiments validated on multiple AR backbones, with long-video quality and VBench-Long ranking both best in class; it is also the first open method to model "self-generated latent representation" as "content-addressable retrieval memory."

Commentary: Migrating RAG from language models to video generation is not a simple analogy — the spatio-temporal continuity of video makes "what to retrieve" a key design point. NVIDIA's solution is very ecosystem-friendly: it does not retrain the base diffusion model, but only adds the retrieval mechanism on the outside layer, a low-cost, backward-compatible improvement for the long-video generation ecosystem. When Sora, Kling, Wan, and other mainstream frameworks are all rolling for longer and more stable, this kind of "plug-in" method may be quickly absorbed into industrial-grade pipelines.