[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-omnicompass-omni-modal-video-grounding-benchmark":3,"topics-all":38,"news-related-26801a16-3f57-4bd8-adad-3b49a1d28f08":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"26801a16-3f57-4bd8-adad-3b49a1d28f08","OmniCompass基准:开源模型音视频理解仍落后","香港理工大学团队发布全模态视频理解基准 OmniCompass:820 段视频、2460 条人工标注实例,从时间定位、空间定位、问答三任务考察 50 个多模态模型的「何时、何地、何事」能力。结果显示开源模型整体落后闭源,音视频联合理解差距最明显,头部模型的时空定位也不够精准。","多模态大模型的视频理解能力到底有几斤几两?香港理工大学 VCLab 团队(VCLab-PolyU)10 月 13 日提交到 arXiv 的基准 OmniCompass(编号 2610.18829)给出了一份系统性体检报告:820 段视频、2460 条人工标注实例、50 个多模态模型同场测试。结论并不乐观——开源模型整体落后闭源,音视频联合理解的差距尤其刺眼。这篇论文也在 10 月 16 日的 Hugging Face Daily Papers 榜单上以 301 个赞排在最前。\n\n## 考的不是「看没看到」,而是「何时、何地、何事」\n\n现有视频理解基准大多只覆盖部分评测维度,难以系统诊断全模态视频理解。OmniCompass 把评测拆成两条赛道:OmniCompass-VL 面向视觉语言模型,只看视觉证据;OmniCompass-AV 面向全模态模型,考察音视频联合证据。每条赛道下设三个互补任务:时间定位(TG,目标何时出现)、空间定位(SG,目标在哪里)、问答(QA,发生了什么)。问答在 VL 赛道覆盖 5 类感知加 3 类推理,在 AV 赛道覆盖 3 类感知加 2 类推理。数据侧,VL 赛道 820 段视频、2460 条实例;从中抽出叙事与音频丰富的子集构成 AV 赛道,240 段视频、720 条实例。视频按时长分短(90 秒以内)、中(90 到 300 秒)、长(300 秒以上)三档,内容横跨七个领域、总时长 80.6 小时。团队在项目主页披露,整个基准投入约 1800 个人工标注小时,每条实例都经多轮质量校验。\n\n## 同一个目标,两种问法\n\n这个基准最巧的设计是「成对指定」:同一段视频里的同一个目标,分别用纯文本、文本加参考图两种方式指给模型。榜单还专门定义了跨指定一致性分数 CS,衡量成对预测是否既正确又相互一致。结果暴露出一个容易被平均分掩盖的问题:模型在两种问法下经常给出不一致的预测——你用文字问它和给它看图,它认的可能不是同一个东西。\n\n## 50 个模型的成绩单:三处硬伤\n\n50 个多模态大语言模型的评测结果呈现三个信号:其一,开源模型整体落后闭源模型,音视频联合理解差距尤其明显;其二,即便头部模型,精确的时间与空间定位依然困难,「大概在这段」和「精确到秒」之间还隔着一道坎;其三,成对指定下的预测不一致普遍存在。换句话说,视频理解要从「看得到」走到「指得准」,行业还有一段路要赶。\n\n## 数据开放,可以直接上手\n\n基准数据集已上架 Hugging Face(VCLab-PolyU\u002FOmniCompass),项目主页 omnicompass.netlify.app 提供 50 个模型的完整榜单与一致性得分,论文以 CC BY 4.0 许可发布。对做多模态方向的开发者,这是一份现成的诊断工具:与其在通用榜单上卷平均分,不如用「何时、何地、何事」三问定位自家模型的真实短板。当视频理解成为 Agent 操作视频界面、具身系统读懂环境的地基,「指得准」也许会比「答得出」更早成为分水岭。\n\n原文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.18829","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.18829","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"53064837-9504-41e8-9904-0a0f67ec3de1","en","OmniCompass Benchmark: 50 MLLMs Tested, Open Models Lag","OmniCompass probes 50 multimodal models on when, where and what in video; open weights trail closed models, widest on audio-visual tasks.","How well do multimodal LLMs actually understand video? OmniCompass, a benchmark submitted to arXiv on October 13 by the VCLab team at Hong Kong Polytechnic University (arXiv:2610.18829), offers a systematic checkup: 820 videos, 2,460 manually verified instances, and 50 multimodal models evaluated side by side. The verdict is sobering — open-source models still trail proprietary ones, and the audio-visual gap is especially wide. The paper also sits at the top of the October 16 Hugging Face Daily Papers list with 301 upvotes.\n\n## Not whether it saw, but when, where, and what\n\nExisting video benchmarks cover only part of the evaluation space, which makes systematic diagnosis hard. OmniCompass splits evaluation into two tracks: OmniCompass-VL tests vision-language models on visual evidence, while OmniCompass-AV tests omni-modal models on joint audio-visual evidence. Each track runs three complementary tasks: temporal grounding (when a target appears), spatial grounding (where it is), and question answering (what happens). QA spans five perception plus three reasoning categories on the VL track, and three perception plus two reasoning categories on AV. The VL track holds 820 videos and 2,460 instances; a narrative- and audio-rich subset forms the AV track with 240 videos and 720 instances. Videos fall into short (under 90 seconds), medium (90 to 300), and long (over 300) buckets, spanning seven domains and 80.6 hours of content. The team reports roughly 1,800 human annotation hours on the project homepage, with every instance passing multi-round quality verification.\n\n## One target, two ways to ask\n\nThe sharpest design choice here is paired specification: the same target in the same video is referenced twice — once with text only, once with text plus a reference image. A Cross-Specification Consistency Score (CS) measures whether paired predictions are both correct and mutually consistent. The results expose a problem that average scores hide: models frequently give inconsistent predictions across the two phrasings — asked in words or shown an image, they do not always recognize the same thing.\n\n## The scoreboard: three weak spots\n\nEvaluations of 50 MLLMs show three signals. First, open-source models lag proprietary counterparts, with the audio-visual gap especially wide. Second, precise temporal and spatial grounding remains challenging even for leading models. Third, inconsistencies across paired specifications are pervasive. In short, the industry still has a way to go from seeing to pointing precisely.\n\n## Open data, ready to use\n\nThe dataset is live on Hugging Face (VCLab-PolyU\u002FOmniCompass), the project homepage omnicompass.netlify.app publishes the full 50-model leaderboard with consistency scores, and the paper is released under CC BY 4.0. For multimodal developers, this is a ready-made diagnostic: instead of chasing averages on general leaderboards, use the when\u002Fwhere\u002Fwhat triad to locate real weaknesses. As video understanding becomes the foundation for agents that operate video interfaces and embodied systems that read environments, pointing accurately may become the dividing line sooner than answering correctly.\n\nSource: https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.18829","omnicompass-omni-modal-video-grounding-benchmark","2026-10-16T15:10:00Z","2026-10-16T15:09:25.618010Z","2026-10-16T15:09:25.618020Z",true,"agent",175,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"5d3c50e8-5087-43e9-a8f1-c64973f712c1","Qwen 拆掉 ASR 管道:音视频原生对话靠合成数据练成","qwen-omnivchat-native-audio-visual-dialogue","2026-09-21T15:14:15+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"ff0bc92a-295a-4707-be8d-76115fe9eeee","PerceptionBench 出炉:16 个前沿多模态模型,视觉感知无一及格","moonshot-perceptionbench-atomic-perception","2026-08-26T13:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"cf7aeaaa-7179-4de2-b177-9adf54668d61","StreamEval：37 模型实测，实时视频最缺听指令","streameval-interactive-streaming-video-benchmark","2026-10-22T11:08:09+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ace7a537-e2de-448f-811f-a028357aed42","FocusGUI:给全模态GUI智能体装上注意力滤镜","focusgui-omni-modal-gui-agents","2026-10-21T19:05:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"aa718f3c-14cf-4984-8762-794eaaee8c16","MAI-Gym:9B 智能体在 2 万个并行环境里做在线 RL","mai-gym-gui-agent-online-rl-9b","2026-10-21T09:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"15da3422-7db5-4da6-a12a-e4fd84b9a50b","视觉脚手架:88M 参数把视觉特征搬进思维链","visual-scaffolding-native-multimodal-reasoning","2026-10-20T23:03:57+00:00"]