[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-vidu-s2-editable-spatial-video":3,"topics-all":38,"news-related-e508c8a9-5355-491d-b489-780ba845533c":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"e508c8a9-5355-491d-b489-780ba845533c","Vidu S2 把实时视频生成拉到 720p:能交互、能剪辑、还能立体","Vidu 团队发布 S2 实时视频生成模型:Avatar 支持 720p、25-42 FPS 实时交互与动态换参考图,Editing 可实时改视频流,还能输出立体视频。论文登顶 Hugging Face 当日热度第一。","9 月 10 日,Vidu 团队把 S2 的技术报告挂上 arXiv(编号 2609.11638),五天后,这篇论文冲上 Hugging Face Daily Papers 当日热度第一,拿到 393 个 upvote。S1 七月才发布,S2 就已经落地——这次升级的密度,值得拆开看。\n\n## 一次升级,三个能力\n\nS2 不再是单个模型,而是两个:S2-Avatar 管实时交互数字人,S2-Editing 管实时视频编辑。数字人这边,分辨率从 S1 的 540p 拉到 720p,帧率 25-42 FPS;更关键的是\"动态参考\"——直播过程中随时丢一张新参考图,就能换装、换道具、换场景,不用重开流。指令跟随也加强了,从\"说话头\"扩展到跳舞这类大肢体动作。S2-Editing 则是对着视频流实时改:风格迁移、虚拟试穿、角色替换、背景替换,同时保住原视频的运动和节奏。还有一步比较超前:生成的或编辑过的画面可以实时转成左右眼同步的立体视频,直接推给 VR 头显。\n\n## 流式生成最怕的\"漂移\",S2 用 Self-Replay Forcing 治\n\n流式视频生成的老大难是误差累积:每一段都以自己上一段的输出为条件,错一点就会滚雪球。S2 的解法叫 Self-Replay Forcing(SRF)——把模型自己生成的轨迹加噪后重放,在梯度可用的因果 pass 里再过一遍,让模型学会\"接住自己的输出\"而不是被它带偏,防止误差跨分段累积。这不是玄学,是对流式生成稳定性的正面攻坚。\n\n## 推理栈才是隐藏主角\n\n720p、25-42 FPS 不是模型单独扛下来的。官方 GitHub(shengshu-ai\u002FVidu-S)写得很清楚:TurboDiffusion 加 TurboServe 的组合,靠高效 attention、低位 GEMM、kernel 与启动优化、多 GPU 流水线,把实时推理塞进低成本 GPU。作者报告称,在他们评测的五个公开基准上,S2 全面超过基线模型——注意这是作者自评,独立复现还没出现,引用时留个心眼。\n\n## 所以呢\n\n从 S1 到 S2,路线图已经很清晰:实时视频生成正在从\"演示品\"变成\"基础设施\"。Avatar 对着数字人直播与陪伴场景,Editing 对着虚拟试穿与影视预演,空间视频对着 VR——每一项都踩在具体应用上。真正的悬念在成本:低位 GEMM 和流水线优化决定了这套东西能不能长期跑在便宜显卡上。可玩 demo 已经开在 vidu.com\u002Fvidu-stream,API 文档同步放出;想验证的话,自己跑一遍比看跑分有用。\n\n参考:[arXiv 2609.11638](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.11638) \u002F [GitHub: shengshu-ai\u002FVidu-S](https:\u002F\u002Fgithub.com\u002Fshengshu-ai\u002FVidu-S)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.11638","f2ab33ad-693b-4d58-8cbd-49498d81c30f",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",{"id":19,"name":20,"slug":20,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":22,"name":23,"slug":23,"description":14,"color":14},"ebe5dcd1-46b1-4298-b8c2-8e0e2f456e56","video-generation",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"60c4456e-3377-4929-95ef-8bd074594992","en","Vidu S2 pushes real-time video generation to 720p: interactive, editable, and spatial","The Vidu team releases S2, a real-time video generation upgrade: S2-Avatar runs 720p at 25-42 FPS with mid-stream reference swaps, S2-Editing rewrites live video flows, and both can output stereoscopic video. The paper topped Hugging Face Daily Papers.","On September 10, the Vidu team posted the S2 technical report on arXiv (2609.11638); five days later it climbed to #1 on Hugging Face Daily Papers with 393 upvotes. S1 only shipped in July, and S2 is already here — the density of this upgrade deserves a closer look.\n\n## One upgrade, three capabilities\n\nS2 is no longer a single model but two: S2-Avatar handles real-time interactive digital characters, while S2-Editing handles real-time video editing. On the avatar side, resolution moves from S1's 540p up to 720p at 25-42 FPS. More important is \"dynamic references\": drop a new reference image at any moment mid-stream to change outfits, props, or scenes without restarting. Instruction following also got stronger, expanding beyond talking heads to large body motions such as dancing. S2-Editing transforms an incoming video stream in real time — style transfer, virtual try-on, character replacement, background replacement — while preserving the source motion and timing. One more forward-leaning step: generated or edited streams can be converted in real time into synchronized left- and right-eye views for stereoscopic video, streamed straight to VR headsets.\n\n## Fighting streaming drift with Self-Replay Forcing\n\nThe chronic pain of streaming video generation is error accumulation: each segment is conditioned on the previous segment's own output, so small errors snowball. S2's answer is Self-Replay Forcing (SRF) — re-noising and replaying the model's self-generated trajectories through a gradient-enabled causal pass, teaching the model to \"catch its own output\" rather than drift with it, and preventing errors from accumulating across streaming segments. This is a head-on attack on streaming stability, not hand-waving.\n\n## The inference stack is the hidden protagonist\n\n720p at 25-42 FPS is not carried by the model alone. The official GitHub repo (shengshu-ai\u002FVidu-S) is explicit: a TurboDiffusion plus TurboServe serving stack combines efficient attention, low-bit GEMM, kernel and launch optimizations, and multi-GPU pipelining to squeeze real-time inference onto low-cost GPUs. The authors report that S2 outperforms all baselines across the five public benchmarks in their evaluation — note this is a self-reported claim, independent reproduction has not appeared yet, so treat it with care.\n\n## So what\n\nFrom S1 to S2 the roadmap is now clear: real-time video generation is graduating from demo to infrastructure. Avatar targets digital-human livestreaming and companionship, Editing targets virtual try-on and film previz, spatial video targets VR — each lands on a concrete application. The real open question is cost: low-bit GEMM and pipelining optimizations decide whether this stack can run long-term on cheap GPUs. A playable demo is live at vidu.com\u002Fvidu-stream with API docs published alongside; if you want to verify, running it yourself beats reading benchmarks.\n\nRefs: [arXiv 2609.11638](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.11638) \u002F [GitHub: shengshu-ai\u002FVidu-S](https:\u002F\u002Fgithub.com\u002Fshengshu-ai\u002FVidu-S)","vidu-s2-editable-spatial-video","2026-09-15T17:05:00Z","2026-09-15T17:06:44.248332Z","2026-09-15T17:06:44.248343Z",true,"agent",66,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"bcdc10bc-2f08-4c39-8ffa-e7e34041c112","京东开源 JoyAI-Video-Edit:用 16B 多模态扩散 Transformer 把视频编辑推进「边播边改」实时流时代","jd-joyai-video-edit-real-time-streaming","2026-08-05T03:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"d1657f4a-aaff-41b3-b66a-20c689775794","Vidu S1 把视频模型带向“实时交互”：AR + Diffusion 在消费级 GPU 上跑通无限时长","vidu-s1-realtime-interaction","2026-07-03T14:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"5612d186-46ee-4509-9a93-94045ba004ae","LTX-2.5 开放权重视频模型:4K 反而在 Fast 端点,EXR 色彩管线也焊进去了","ltx-2-5-open-weights-video","2026-08-18T15:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"6f9e9f94-9dcc-4c6c-b254-6c5d0fe8ed37","京东开源 JoyAI-Video-Edit:16B 多模态扩散 Transformer 把视频编辑推进「边播边改」实时流时代","jd-joyai-video-edit-realtime-diffusion","2026-08-10T00:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"d3e01f3d-745b-4c98-9289-38081a3f5f06","FLUX 3：图像\u002F视频\u002F音频统一进 flow matching","bfl-flux-3-flow-matching","2026-07-27T10:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"caed836e-2168-418f-b5c1-bde3ce962e66","Mask Forcing 往蒸馏 rollout 里掺干净 token:修视频生成的模式坍缩,指令遵循最高涨 6.5 分","mask-forcing-video-diffusion-distillation","2026-09-09T23:08:37+00:00"]