[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-deltastreamer-streaming-video-replay":3,"topics-all":38,"news-related-601ab1a1-6704-4bd9-bd4e-81fe87366a25":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"601ab1a1-6704-4bd9-bd4e-81fe87366a25","DeltaStreamer开源:流式视频理解学会\"回头补证据\"","南开大学、字节跳动等机构开源流式视频理解框架 DeltaStreamer:持续观察视频流,把画面变化写进带实体索引的结构化记忆;提问后回放历史帧补齐缺失证据。论文报告其在 OVO-Bench、StreamingBench 总分 66.07%、59.53%,超过代表性方法。","先看一个别扭的设定:视频在实时流式地来,而问题可能下一秒才到。等你听到\"刚才那个人穿的外套是什么颜色\",那一帧早就过去了。流式视频理解的核心难题从来不是看得不够清,而是必须在不知道问题是什么的前提下,提前决定记住什么。现有做法大多把视频历史压缩成摘要,既容易抹掉事件转折点,也无法在事后找回被丢掉的细节。\n\n南开大学 VCIP、字节跳动、IQuest Research 与中科院自动化所的研究者开源了 DeltaStreamer(arXiv:2610.19849),给这个问题补上了一个人类本能的动作:回头去看。\n\n## 先记变化,再回头查证\n\nDeltaStreamer 是一个面向流式视频理解的 agentic 框架,由三块组成:\n\n- **显式视觉增量**:逐帧对比相邻画面,把\"变化了什么\"与场景描述一起保留,让事件转折显式化;\n- **结构化文本记忆**:把观察结果、片段摘要和钉住的事实组织成分层 DAG,带实体索引和指向原始证据的指针;\n- **查询驱动规划与重放**:问题到来后,先规划回答需要哪些证据,引导检索与工具调用;记忆不够时,定位更早的场景并回放历史帧,把文本记忆里丢掉的视觉细节直接读回来。\n\n一句话概括:观察视频流、记住发生了什么、记忆不够用时回去查证据。整个系统无需额外训练,搭配本地 VLM 权重即可运行。\n\n## 跑分与工程细节\n\n论文报告的总分:OVO-Bench 66.07、StreamingBench 59.53,超过代表性流式理解方法——注意这是论文自报数字,README 同时明确说明,开源版本尚未完成完整的 GPU 推理回归。工程栈相当\"能跑\":vLLM 0.19.1、ms-swift 4.2.0 加 transformers 5.5.4;主干用 Qwen3.6-27B 负责观察、规划与回答,SAM2.1 Hiera Large 做目标追踪,Whisper base 做语音转写。仓库附带中英双语界面的交互 demo,上传视频、选时机提问,可以看到 agent 的观察、记忆与计划全过程。许可证为 CC BY-NC 4.0,商用需另谈;训练代码与实验工件未放出,本次发布只保留推理实现、prompts 与记忆公式。\n\n## 为什么值得盯\n\n这个工作代表了一条与\"端到端模型内建记忆\"不同的路线:不赌模型把一切都装进上下文,而是把\"记什么、何时回头\"做成显式的策略层。代价是重放引入额外延迟与存储,但对直播这类允许\"稍后补答\"的场景,换来的证据级准确性是划算的——团队自己也顺着这条路径发布了直播陪伴应用 AI 同担,连弹幕建议都做了。对做视频监控、直播理解、多模态 agent 的团队,这套\"变化感知记忆 + 定向重放\"的拆法值得拆开细读:当看视频的 AI 从被动压缩转向主动调查,流式理解的瓶颈可能不在模型大小,而在记忆策略。\n\n参考:arXiv:2610.19849(https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.19849)· 代码:https:\u002F\u002Fgithub.com\u002FBBBBchan\u002FDeltaStreamer","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.19849","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"da116f20-377f-449f-9382-821d56f6fa64","streaming",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"7d897d89-add4-4073-9039-8dc0259c6a41","en","DeltaStreamer: Streaming Video AI Learns to Replay Evidence","DeltaStreamer logs visual deltas in structured memory and replays frames to fill evidence gaps. Paper-reported: 66.07% OVO-Bench, 59.53% StreamingBench.","Consider an awkward setup: video arrives as a live stream, but the question may come a second later. By the time someone asks \"what color was that person's jacket?\", the frame is long gone. The core problem of streaming video understanding is not seeing clearly — it is deciding what to remember before knowing what will be asked. Most existing approaches compress video history into summaries, which blurs event transitions and cannot recover dropped details afterwards.\n\nResearchers from VCIP at Nankai University, ByteDance, IQuest Research and the Institute of Automation at the Chinese Academy of Sciences have open-sourced DeltaStreamer (arXiv:2610.19849), which adds a deeply human move to this problem: looking back.\n\n## Record changes first, then revisit\n\nDeltaStreamer is an agentic framework for streaming video understanding, built from three parts:\n\n- **Explicit visual deltas**: consecutive frames are compared, and what changed is retained alongside scene descriptions, making event transitions explicit.\n- **Structured textual memory**: observations, episode summaries and pinned facts are organized into a hierarchical DAG with entity indexing and pointers to source evidence.\n- **Query-driven planning and replay**: once a question arrives, the planner identifies what evidence the answer needs and guides retrieval and tool use; when memory falls short, it locates earlier scenes and replays historical frames to read back visual details that textual memory dropped.\n\nIn one line: observe the stream, remember what happened, and revisit the visual evidence when memory falls short. The system requires no additional training and runs with a supported local VLM checkpoint.\n\n## Scores and engineering details\n\nPaper-reported overall scores: 66.07 on OVO-Bench and 59.53 on StreamingBench, improving over representative streaming video understanding methods — note these are paper-reported numbers, and the README states plainly that the open-source release has not yet completed full GPU inference regression. The engineering stack is refreshingly runnable: vLLM 0.19.1, ms-swift 4.2.0 and transformers 5.5.4; a Qwen3.6-27B backbone handles observation, planning and answering, SAM2.1 Hiera Large does object tracking, and Whisper base handles audio transcription. The repository ships an interactive demo with English and Chinese interfaces: upload a video, pick when to ask, and inspect the agent's observations, memory and plans. The license is CC BY-NC 4.0, so commercial use needs separate terms; training code and experiment artifacts are excluded — this release preserves the inference implementation, prompts and memory formulas.\n\n## Why it deserves attention\n\nThis work represents a different route from \"end-to-end models with built-in memory\": instead of betting the model can hold everything in context, it makes \"what to remember and when to look back\" an explicit policy layer. The cost is extra latency and storage from replay, but for livestream-style settings where a slightly delayed answer is acceptable, evidence-grounded accuracy is a good trade — the team followed this exact path with AI-Tongdan, a livestream companion app that even suggests danmaku comments. For teams working on video monitoring, livestream understanding or multimodal agents, this decomposition of change-aware memory plus targeted replay is worth a close read: when watching AI shifts from passive compression to active investigation, the bottleneck of streaming understanding may not be model size, but memory strategy.\n\nReferences: arXiv:2610.19849 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.19849) · Code: https:\u002F\u002Fgithub.com\u002FBBBBchan\u002FDeltaStreamer","deltastreamer-streaming-video-replay","2026-10-21T17:11:14Z","2026-10-21T17:11:22.138119Z","2026-10-21T17:11:22.138129Z",true,"agent",56,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"ace7a537-e2de-448f-811f-a028357aed42","FocusGUI:给全模态GUI智能体装上注意力滤镜","focusgui-omni-modal-gui-agents","2026-10-21T19:05:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"63ae3eda-4994-4f9f-a812-489d9831a9ca","Clef-omni 开源:决策模型学会听音看片","clef-omni-multimodal-decision-model","2026-10-12T15:20:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"d5ba4b97-e08e-4ddb-80d8-70f8060f6d9a","AgentGarten:把世界写成代码,智能体4轮学会躲猫猫","agentgarten-code-worlds-for-evolving-agents","2026-10-10T17:10:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"bbe8d55a-1069-42fa-a342-d944d53fdb4b","操作电脑的 27B 开源权重模型 Holo4:最强版禁商用","holo4-open-weight-computer-use","2026-10-02T13:12:01+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"dc61debd-1d77-4d5a-9d59-5b23c3da07de","蚂蚁开源Realtime-Venus：9B全双工模型边说边干活，三项续聊指标超GPT-4o","ant-realtime-venus-full-duplex-delegation","2026-09-30T23:10:53+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"089f56f3-32ff-4036-89b5-728d5f5a9359","边聊边干活:腾讯混元开源全模态交互 Agent Gander,小脑管对话、大脑管执行","hunyuan-gander-omni-interaction-agent","2026-09-09T21:07:00+00:00"]