Video generation models keep getting stronger, but as soon as the output crosses a few dozen frames, geometric drift and object "face-swapping" become routine — the root cause is that flattening the entire video and feeding it to a Transformer makes the sequence length explode exponentially. MIT's Sitzmann lab (Ishaan Preetam Chandratreya, David Charatan, Basile Van Hoorick et al.) released MilliVid on arXiv on June 8, splitting the problem into two pieces. First, a multi-scale autoencoder. Each frame is compressed into a set of hierarchical tokens — from the regular latent variables all the way down to "a few tokens per frame." The coarsest level retains only scene layout, semantics, and object identity; the fine levels then supplement high-frequency appearance and texture. So information of different importance naturally stratifies. Second, coarse-to-fine rollout video diffusion. During training the coarse tokens are generated first, then used to guide the generation of fine tokens; at inference the model only imposes long-horizon consistency constraints on the dimensions "worth allocating compute to." The result is significantly better geometry and object persistence than existing baselines on long Minecraft video evaluation. Compared to cramming context into attention or breaking long-range dependencies with "sliding windows," MilliVid's approach is closer to human vision — first remember the skeleton, then fill in the details. The lesson for industry is: long video generation doesn't necessarily need 1M context; using "coarse to fine" to allocate compute is more effective than stacking length.