[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-prism-2k-video-audio-sparse-attention":3,"topics-all":38,"news-related-0b0af156-7b5c-47d8-bfc2-de8cc0b22ce2":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"0b0af156-7b5c-47d8-bfc2-de8cc0b22ce2","混元开源Prism:稀疏注意力让2K音视频训练提速2.5倍","腾讯混元联合复旦、浙大开源 Prism 框架,以动态稀疏注意力原生训练 2K 音视频一体生成模型:训练提速 2.5 倍,质量反超全注意力。代码与 preview 权重以 MIT 许可放出,支持 720p\u002F1080p\u002F2K 三档联合生成,音频可用 music\u002Fsfx\u002Fspeech 标签结构化控制。","视频生成模型的竞争焦点正在从\"能动\"滑向\"高清\":720p 逐渐成为入场配置,2K 才是下一个战场。但分辨率每上一个台阶,Transformer 的全注意力成本就按平方倍数膨胀一次;更隐蔽的代价是,注意力会摊到越来越多的冗余 token 上,真正值得学的信号反被稀释。腾讯混元联合复旦大学、浙江大学 10 月 4 日提交到 arXiv 的 Prism 框架([arXiv:2610.05416](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.05416)),把稀疏注意力从推理加速技巧升级成训练阶段的原生结构,专治音视频一体生成的 2K 训练账单。\n\n## 音画一体生成,注意力却在一视同仁\n\n音视频联合生成指模型在一次去噪过程中同时产出画面和声音。论文的出发点是一个容易被忽略的结构性事实——在联合音视频数据里,跨模态交互天然集中在\"发声区域\",那里才是音频真正施加影响的区域。而全注意力对此一视同仁,把算力平摊到整个时空体积;分辨率升到 2K 后 token 数量激增,冗余 token 摊薄了学习信号,还会扰动预训练先验。论文指出,现有稀疏注意力方案大多瞄准免训练的推理加速,没有针对音视频数据的区域化结构设计。\n\n## 让注意力块的形状跟着内容走\n\nPrism 的核心思路是\"按内容下菜\"。它先把 token 序列组织成时空宏区,让注意力结构能够适配局部内容;再对每个宏区估计两路信号——视频特征沿通道方向的方差,以及音频到视频交叉注意力的特征范数:前者刻画视觉内容在哪些方向上变化剧烈,后者度量音频对每个视觉区域的影响力强弱。基于这两路信号,Prism 给每个宏区动态分配定制的注意力块形状,视觉变化快的轴、音画耦合强的轴切分得更细,块内 token 因此保持语义连贯,块级特征既能概括视觉内容,也能捕捉音视频交互模式。最后再配合混合块选择策略,逐 query 决定稀疏度。\n\n## 提速 2.5 倍,质量不降反升\n\n结果是最有说服力的部分:相比全注意力,Prism 训练提速 2.5 倍,生成质量不降反升。代码与 preview 权重已按 MIT 许可开源([GitHub:Tencent-Hunyuan\u002FPrism](https:\u002F\u002Fgithub.com\u002FTencent-Hunyuan\u002FPrism)),截至撰稿已收获 300 余 star。preview 模型支持 720p、1080p、2K 三档原生音视频联合生成与训练,音频侧可用 `\u003Cmusic>`、`\u003Csfx>`、`\u003Cspeech>` 结构化标签控制;alpha 版本主打稳定,beta 版本主打运动,更强的 Prism-pro 还在路线图上。\n\n## 门槛与看点\n\n先泼冷水:仓库对硬件门槛写得很直白——训练 720p 至少 32 张 80GB GPU,1080p\u002F2K 至少 64 张;推理 720p 单卡可跑,2K 需要 4 张以上,目标用户是有集群的研究团队,不是消费级玩家。\n\n还有两点值得注意。其一,训练端稀疏与推理端稀疏是两条互补的线:业界此前更多做推理侧免训练稀疏加速,救的是部署成本;Prism 把稀疏做进训练本身,救的是训练预算,两者可以叠加。其二,音视频一体生成正在成为开源赛道的明牌,MIT 许可加上数据预处理、分桶、FSDP 训练、推理脚本的完整管线全部放出,延续了混元系在视频生成上的开源节奏。\n\n所以,分辨率竞赛的下一程,瓶颈未必是更大的集群,而是更聪明的注意力结构。当 2K 训练的注意力账单能砍下大半,\"原生高清\"才从大厂特权变成可复制的工程问题。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.05416","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"ebe5dcd1-46b1-4298-b8c2-8e0e2f456e56","video-generation",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f7385465-1682-4521-b7c0-91b8b606955f","en","Prism: Sparse Attention Cuts 2K Video-Audio Training 2.5x","Hunyuan open-sources Prism: sparse attention trains 2K joint video-audio models 2.5x faster than full attention, with better quality. MIT license.","The frontier of video generation is sliding from \"it moves\" to \"it is sharp\": 720p is becoming the entry ticket, and 2K is the next battleground. But every step up in resolution inflates the quadratic cost of full attention in the Transformer; the subtler damage is that attention spreads over more and more redundant tokens, diluting the learning signal that actually matters. Prism, a framework submitted to arXiv on October 4 by Tencent Hunyuan together with Fudan University and Zhejiang University ([arXiv:2610.05416](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.05416)), upgrades sparse attention from a common inference-acceleration trick into a native structure for training, aimed squarely at the 2K training bill of joint video-audio generation.\n\n## Joint video-audio generation, attention that treats everything alike\n\nJoint video-audio generation means the model produces picture and sound in a single denoising pass. The paper starts from a structural fact that is easy to miss: in joint video-audio data, cross-modal interactions are inherently concentrated around sound-producing regions — that is where audio actually exerts influence. Full attention treats all of this alike, spreading compute across the entire spatiotemporal volume; once resolution climbs to 2K and token counts explode, redundant tokens dilute the learning signal and disrupt pretrained priors. The authors also note that existing sparse-attention methods mostly target training-free inference acceleration, and none of them are designed around this region-based structure of joint video-audio data.\n\n## Let the attention block shape follow the content\n\nPrism's core idea is to partition attention by content. It first organizes the token sequence into spatiotemporal macro-zones, so the attention structure can adapt to local content. For each zone it estimates two signals: the variance of video features along the channel dimension, and the feature norms from audio-to-video cross-attention — the former captures where visual content changes sharply, the latter measures how strongly audio influences each visual region. From these signals, Prism dynamically assigns a tailored block shape to every zone, applying finer partitioning along axes of rapid visual variation and strong audio-visual coupling. Tokens inside each block stay semantically coherent, so block-level features capture both visual content and joint video-audio interaction patterns. A hybrid block-selection strategy then decides per-query sparsity.\n\n## 2.5x faster training, and quality goes up\n\nThe headline result: versus full attention, Prism achieves a 2.5x training speedup while surpassing it in generation quality. The code and preview weights are open-sourced under MIT ([GitHub: Tencent-Hunyuan\u002FPrism](https:\u002F\u002Fgithub.com\u002FTencent-Hunyuan\u002FPrism)), with 300+ stars at the time of writing. The preview model supports native joint video-audio generation and training at 720p, 1080p, and 2K, with structured audio control via `\u003Cmusic>`, `\u003Csfx>`, and `\u003Cspeech>` tags; the alpha checkpoint targets stability, the beta targets motion, and a stronger Prism-pro is on the roadmap.\n\n## Hardware bar, and what to watch\n\nCold water first: the repository is blunt about hardware. Native 720p training needs at least 32 NVIDIA 80 GB GPUs, and 1080p\u002F2K each need at least 64; for inference, 720p runs on a single 80 GB card while 1080p and 2K require 4 or more. This is a framework for research teams with clusters, not a consumer toy.\n\nTwo things are worth noting. First, training-side sparsity and inference-side sparsity are complementary tracks: the industry has mostly pursued training-free sparse acceleration at inference time to save deployment cost; Prism bakes sparsity into training itself to save training budget — the two can stack. Second, joint audio-video generation is becoming an open race, and the MIT license plus the full pipeline (data preprocessing, bucketing, FSDP training, inference scripts) continues Tencent Hunyuan's open-source cadence in video generation.\n\nSo the next leg of the resolution race may hinge not on bigger clusters but on smarter attention structures. Once the attention bill for 2K training can be cut by more than half, \"natively high-resolution\" stops being a mega-lab privilege and becomes a replicable engineering problem.","prism-2k-video-audio-sparse-attention","2026-10-13T23:13:00Z","2026-10-13T23:13:14.688313Z","2026-10-13T23:13:14.688321Z",true,"agent",243,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"24d56562-4886-4169-8aa9-773f5fa34e2f","GRACE 把视频潜空间压 8 倍:提速 11 倍,跑分反超","grace-latent-compression-video-diffusion","2026-10-08T15:10:59+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"caed836e-2168-418f-b5c1-bde3ce962e66","Mask Forcing 往蒸馏 rollout 里掺干净 token:修视频生成的模式坍缩,指令遵循最高涨 6.5 分","mask-forcing-video-diffusion-distillation","2026-09-09T23:08:37+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"02c8b500-ec11-44a6-8c58-6e880563dad8","FastH3 开源:4 步蒸馏版 MiniMax H3,B200 单卡最高提速 14 倍","fasth3-4-step-distilled-minimax-h3","2026-08-30T21:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"18d2aa73-7244-4b10-b611-46475e17327e","ForgeWM开源:一步去噪72FPS的可玩世界模型,8张卡复现全流程","forgewm-few-step-playable-world-model","2026-08-24T21:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"0599b775-ac17-49d2-aebd-a16f531c7168","腾讯混元 MeanFlowNFT：把 RL 接进「平均速度生成器」，Wan 2.1 4 步反超 50 步 LongCat-Video RL","tencent-hunyuan-meanflownft","2026-07-16T12:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"42cfc778-8f1b-4bf2-a0ae-4343a066f48d","RhymeFlow：清华提出异步去噪流调度，DiT视频生成训练免费加速1.53倍","rhymeflow-tsinghua-async-denoising-1-53x","2026-06-07T22:00:00+00:00"]