SANA-Video 2.0: hybrid linear attention, single-GPU video
·96 views
Bottom line first Video generation's real deployment blocker isn't the picture quality — it's the attention compute each generation has to pay. SANA-Video 2.0, posted to arXiv on July 23, doesn't keep stacking parameters — it changes the attention mechanism into a "hybrid" one: most layers use gated linear attention to keep linear scaling under long sequences, and every few layers insert a gated Softmax attention to recover the global relationships that pure linear attention tends to drop. The two alternate at a 3:1 ratio, paired with Block Attention Residuals, which pass the high-quality global information from the minority of layers on to the layers that follow. The paper provides two scales, 5B and 14B, targeting up to 720p video generation on a single GPU. In experiments, the 5B version generates 720p, 5-second video on a single H100 in 13.06 seconds after full optimization — a 3.2x speedup in compiled DiT forward versus a matched all-Softmax baseline, climbing to 3.58x after operator fusion, caching, and sparse attention stack on top. The authors report a VBench score of 84.30. My take: the value of this paper isn't in the "120x" kind of number that's easy to repost — it's in designing the model structure and the inference system together. Linear attention, viewed in isolation, often sacrifices expressiveness; pure Softmax is dragged down by long-video token counts. SANA-Video 2.0's choice to periodically re-supplement global information is essentially finding an engineering-friendly middle ground between quality and cost. The next round of video-model competition may not be about whose demo is more stunning, but about who can push the wait-time of a single generation into a real workflow. The paper is still a preprint, the metrics depend on specific hardware and optimization stacks, and they can't be equated to speedup in every scenario. But the direction is clear: for video generation to become widespread, attention compute has to be saved first.