[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-streameval-interactive-streaming-video-benchmark":3,"topics-all":38,"news-related-cf7aeaaa-7179-4de2-b177-9adf54668d61":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"cf7aeaaa-7179-4de2-b177-9adf54668d61","StreamEval：37 模型实测，实时视频最缺听指令","论文对照十余个已有视频基准后提出 StreamEval：300 个用例、时长 15 到 300 秒、四维度十个指标，覆盖对话与表演数字人、世界生成、指令与参考图编辑。37 个模型实测显示画质与听指挥能力并不同步，听指令最高分 36.90，参考图编辑最高成功率仅 15.00。","实时视频生成正在从「生成一段片段」转向「边生成边接受用户指令」。但当请求在会话进行中不断到来，问题就不再是单帧好不好看，而是模型能不能按时机执行变更、同时不破坏已经生成的内容。论文给出的判断很直接：视觉质量高，不等于听指挥。\n\n## 一个此前没人系统测过的空白\n\n论文逐一对照了 VBench、T2V-CompBench、TC-Bench、NarraStream-Bench、MBench、WorldMark、WBench、StreamAV-Bench、GenBench、THEval、Full-Duplex-Bench、VideoFDB、EditVerseBench 等十余个已有基准，结果是没有任何一个同时覆盖全部五类交互场景。作者据此把 StreamEval 定位为首个统一的交互式流式视频生成基准，榜单在 streameval.ai 开放，代码 Apache 2.0、数据 CC BY 4.0，Hugging Face 与 ModelScope 提供约 3.1 GB 数据集。\n\n## 300 个用例，四个维度，十个指标\n\n基准包含 300 个脚本化测试用例，单次会话时长 15 到 300 秒。每个用例规定初始素材、逐步到达的多模态请求，以及明确的状态变更与内容保持要求。评测分任务执行、状态一致性、视觉保真度、音视频协同四个维度，落到十个 0 到 100 分制指标。时长分档上每档 12 个用例、每个模型每个用例跑三次独立运行，五个时长档在任务内等权聚合，长会话不会因为时间更长而占据更大权重。\n\n## 37 个模型的名次：强者也会失手\n\n数字人表演任务里，Vidu S2 Avatar 以 99.60 的音视频同步和 97.25 的稳定性领先，综合分 81.71。但即便在这样的系统上，听指令执行（Following）也只拿到 36.90，而该项最低值是 Avatar-Forever 22B 的 6.37。论文特意点出：有些系统身份保持超过 80 分、稳定性超过 96 分，听指令这一项依然很低。\n\n世界生成的分歧更明显。综合分上 MemFlow 以 33.36 略高于 Rolling Forcing 的 33.01；但换到 Following 一项，Krea Realtime 25.93 排第一，综合分领先者并非指令执行领先者。HappyOyster 1.0 Adventure 的画质与稳定性最高（58.34 和 56.67），Following 却只有 5.56、连续性 2.78。ShotStream 画质 33.80，稳定性和连续性分别掉到 20.55 和 9.34。\n\n最薄弱的是参考图编辑：在报告该项的三个模型里，最高编辑成功率为 15.00 分。\n\n## 比「更好看」更难的那部分\n\n论文标注的机构包括清华大学、NVIDIA、CMU、UC Berkeley、普林斯顿、新加坡国立大学等，提交后登上 Hugging Face 当日论文第三。\n\n这套评测的价值恰恰在于它不奖励「画面更真」：一个模型可以在视觉指标上拿高分，同时在收到「把镜头转向左边」这类请求时基本不响应。论文结论也承认了边界——自动评判依赖预设的期望结果，下一步要收集人工两两判断并检验其与评分规则的一致性。\n\n所以下次再看到实时视频模型的演示，值得多问一句：它是在播放一段录好的视频，还是真的在听你说话。榜单目前给出的答案是，前者的画质早就跑完了，后者的执行还在及格线上挣扎。\n\n原文：[arXiv:2610.28402](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28402)，代码与榜单见 [github.com\u002FStreamEval\u002FStreamEval](https:\u002F\u002Fgithub.com\u002FStreamEval\u002FStreamEval)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28402","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"ebe5dcd1-46b1-4298-b8c2-8e0e2f456e56","video-generation",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"d49b8f98-4236-474d-ae43-516ce1af82e0","en","StreamEval: 37 Models Tested, Realtime Video Listens Poorly","StreamEval benchmarks interactive streaming video generation: 300 scripted cases across five tasks, with 37 models tested on request execution and state.","Streaming video generation is moving from producing a clip toward accepting user instructions mid-generation. But once requests keep arriving during a live session, the hard question is no longer whether individual frames look convincing. It is whether a model can execute each change at the right moment while leaving already-generated content intact. The verdict from a new benchmark is blunt: high visual quality does not equal following instructions.\n\n## A gap nobody had measured systematically\n\nThe paper compares itself against more than a dozen existing video generation and interactive benchmarks, including VBench, VBench++, T2V-CompBench, TC-Bench, NarraStream-Bench, MBench, WorldMark, WBench, WBench-Editing, StreamAV-Bench, GenBench, THEval, Full-Duplex-Bench, VideoFDB and EditVerseBench. None of them covers all five interactive scenarios at once, so the authors position StreamEval as the first unified benchmark for interactive streaming video generation. The leaderboard is live at streameval.ai, the code is Apache 2.0, the benchmark data is CC BY 4.0, and roughly 3.1 GB of assets are distributed through Hugging Face and ModelScope.\n\n## 300 cases, four dimensions, ten metrics\n\nThe benchmark contains 300 scripted test cases, each session lasting between 15 and 300 seconds. Every case specifies the initial assets, the multimodal requests that arrive incrementally, and explicit requirements about state changes and content preservation. Performance is scored along four dimensions: task execution, state consistency, visual fidelity and audiovisual coordination, instantiated through ten task-specific metrics on a 0 to 100 scale. Sessions are split across duration strata with 12 cases each, and every case is run three times per model, so longer sessions do not dominate the final score.\n\n## Where 37 models actually stand\n\nIn the avatar performance task, Vidu S2 Avatar leads with 99.60 on audiovisual synchronization and 97.25 on stability, for an overall of 81.71. Even so, its Following score reaches only 36.90, while the lowest value in that column belongs to Avatar-Forever 22B at 6.37. The paper highlights the point directly: some systems exceed 80 on identity retention and 96 on stability, and still score poorly on following instructions.\n\nWorld generation is more divided. On overall score, MemFlow takes 33.36, just ahead of Rolling Forcing at 33.01. But on Following the order changes: Krea Realtime leads with 25.93, so the overall leader is not the instruction-following leader. HappyOyster 1.0 Adventure posts the best quality and stability at 58.34 and 56.67, yet its Following drops to 5.56 and continuity to 2.78. ShotStream reaches 33.80 on quality but falls to 20.55 on stability and 9.34 on continuity.\n\nThe weakest area is reference-based editing: among the three models reporting that metric, the highest edit success rate is 15.00 on the same 0 to 100 scale.\n\n## The part that is harder than looking better\n\nThe listed affiliations include Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton and the National University of Singapore, and the paper reached third place on the Hugging Face daily list after submission.\n\nThe value of this evaluation lies in what it refuses to reward. A model can score near the top on visual metrics while largely ignoring a request such as turning the camera left. The authors also acknowledge the boundary of their own work: automatic judging relies on predefined expected outcomes, and collecting human pairwise judgments to check agreement with the scoring rubric is the natural next step.\n\nSo the next time a realtime video model is demoed, it is worth asking one more question: is it playing back recorded footage, or is it actually listening? On the current leaderboard, the first is long since solved, while the second is still struggling to pass.\n\nSource: [arXiv:2610.28402](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28402), with code and leaderboard at [github.com\u002FStreamEval\u002FStreamEval](https:\u002F\u002Fgithub.com\u002FStreamEval\u002FStreamEval) and https:\u002F\u002Fstreameval.ai","streameval-interactive-streaming-video-benchmark","2026-10-22T11:08:09Z","2026-10-22T11:08:44.265876Z","2026-10-22T11:08:44.265883Z",true,"agent",27,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"abd2ad4a-d732-41f0-b1a1-5711103ef7a6","Orbia 基准:生成视频还构不成一致的 3D 世界","orbia-3d-consistency-benchmark","2026-10-18T15:12:16+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"d0d8e772-57f7-47a0-8fd8-dce09f3bd1e5","浙大ATAS登顶HF论文榜:给AI视频装上持久世界记忆","atas-world-state-agentic-video-generation","2026-10-15T15:15:45+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"ba0ed7bf-3de3-4f92-98fe-a50d6ac274d0","WROP 开源:用 150 个物体恒存任务给世界模型补认知课","wrop-object-permanence-world-models","2026-09-25T17:08:02+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"b95b93e8-294a-4c5b-b53d-ce6ea07c1519","SemComp-Bench 登顶 Hugging Face 日榜:视频生成开始考「任务做没做成」","semcomp-bench-video-task-completion","2026-08-20T13:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9fa15063-58b1-4b87-a0cb-0e19ffc4dc6c","4B 参数横扫四大具身基准：开悟世界模型让小模型重新定义 SOTA","kaiwu-4b-world-model-sensetime-72x","2026-06-12T04:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"bef7a773-36fb-406b-949d-8d728b97b242","Matrix-Game 4.0 开源:单卡 60FPS 的实时可玩游戏世界","matrix-game-4-0-open-source-60fps","2026-10-21T13:09:32+00:00"]