[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-opends-world-benchmark-deep-search-agents":3,"topics-all":38,"news-related-205bd813-42eb-4d65-8e8c-5a9af0d8d097":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"205bd813-42eb-4d65-8e8c-5a9af0d8d097","12个模型搜遍真实网页,最高分39%:人类21.1%更低","OpenDS-World 基准让 12 个前沿模型在真实开放网络上做 200 道深度检索题，证据藏在网页、PDF、音视频甚至需手动操作的虚拟漫游里。最高分 GPT-6 Astra 只有 39.0%，仍错 122 题；人类 1000 次独立尝试只有 21.1%，且 42.5% 的题目无人做对。","现在几乎每家厂商都在讲「深度研究」能力：给一个模糊问题，Agent 自己搜网页、翻论文，最后交一份报告。但有个绕不开的问题——凭什么判断它做对了没有？\n\n静态检索基准的答案是「答案在库里，你能不能找出来」。真实网络不是这样：证据可能一半在网页正文、一半在视频字幕、还有一半藏在虚拟博物馆漫游里。\n\n## 把「找证据」本身当考题\n\n论文提出 OpenDS(Open-Web Deep Search)任务，并构建 OpenDS-World 基准：200 道题，覆盖八个领域，全部经独立人工验证以保证答案唯一。真实网络的复杂度被拆成三类：**多模态**（文本、图像、音频、视频）、**异构格式**（HTML、PDF、MP3、MP4、Parquet），以及**交互式访问**——有些证据必须靠翻页、操作图形界面才能拿到。\n\n## 最高 39.0%，还错 122 题\n\n12 个前沿模型在统一工具环境下评测：GPT-6 Astra 以 39.0%（200 题对 78 题）居首，Claude Opus 5.5 为 38.0%、Gemini 3.8 Flash 为 36.5%；Qwen3.8-Flash 12.0% 与 DeepSeek-V4-Pro 12.5% 垫底。**即便是第一，也错了 122 题。**\n\n难度落差更陡。12 个模型平均分从低难度 39.66% 掉到中难度 23.50%、高难度 16.76%。高难度上最高的是 Gemini 3.8 Flash（28.24%），Qwen3.8-Flash 只有 4.71%。\n\n## 人类 21.1%，低于最高分模型\n\n1000 次独立解题尝试中，人类参与者整体准确率 21.1%，低于得分最高的模型；且有 42.5% 的题目，**所有被分配的人类参与者都没做出来**。这不等于「模型超越人类研究者」——两边工具条件与预算未必对等。但它说明：跨多源、多模态做多跳取证时，连人类也会大面积折戟。\n\n## 花更多 token 不等于更准\n\nGPT-6 Astra 平均每题 51.05 次工具调用、18.66 分钟；GPT-5.6 Sol 调用次数几乎相同（52.42 次），只用 8.99 分钟，准确率却低 11 个百分点。Gemini 3.8 Flash 的 token 消耗约为 GPT-6 Astra 的 5.22 倍，准确率反而更低。\n\n## 真正贵的不是搜索，是失败\n\n**工具调用失败只占全部调用的 2.83%–9.42%，却吃掉 19.27%–61.90% 的工具执行时间。** GPT-6 Astra 是 6.34% 的调用数对应 42.57% 的耗时。\n\n预算是 1 小时加 100 次调用双重限制，触顶比例从 Claude Opus 5.5 的 16.5% 到 Qwen3.8-Flash 的 81.5%。失败后 Claude、Gemini、GPT-5.6 Sol 多在 79%–83% 的情况下改用别的工具，Kimi K3 与 GLM-5.3 则有约 37%–38% 改参数重试。前者在换路，后者更像原地打转。\n\n## 所以呢\n\n这份评测（[arXiv:2610.30181](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30181)）的价值不在那个 39.0%，而在于它把「Agent 做深度研究」拆成三件可测量的事：**能不能选对信源、协调不同工具、在预算耗尽前停对。**\n\n前两件决定上限，第三件决定成本曲线。平均 18 分钟的调研 Agent，和跑 45 分钟还交不出答案的 Agent，在定价上不该算同一种东西。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30181","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"de75dbbc-094a-44bc-8c34-b30190335cca","en","12 Models Searched the Real Web, Best Scored 39%","12 frontier models run 200 deep-search tasks on the live Web, where evidence hides in pages, PDFs, and tours. Best score 39.0%, humans 21.1%.","Nearly every vendor now advertises \"deep research\" ability: hand the agent a vague question and it searches the web, reads papers, and returns a report. But one question is hard to dodge: how do you know it actually got the right answer?\n\nStatic retrieval benchmarks have an easy answer: the answer is in the corpus, can you find it. The real web is not like that. The evidence for one question might sit half in page text, half in video subtitles, and half inside a virtual museum tour nobody tells the agent to look at.\n\n## Making evidence acquisition itself the exam\n\nThe paper introduces the OpenDS (Open-Web Deep Search) task and builds the OpenDS-World benchmark on top of it: 200 tasks spanning eight domains, each independently human-verified so that answers are unique. The complexity of the open web is broken into three categories: **diverse modalities** (text, images, audio, video), **heterogeneous formats** (HTML, PDF, MP3, MP4, Parquet), and **interactive access requirements** — some evidence can only be obtained by paging, entering parameters, or driving a graphical interface.\n\n## Best score 39.0%, and still 122 wrong\n\nTwelve frontier models were evaluated under a unified tool environment. GPT-6 Astra took first place at 39.0% (78 of 200 tasks correct), followed by Claude Opus 5.5 at 38.0% and Gemini 3.8 Flash at 36.5%; Qwen3.8-Flash at 12.0% and DeepSeek-V4-Pro at 12.5% came last. **Even the winner got 122 tasks wrong.**\n\nDifficulty hurts more than the headline number suggests. Averaged across the 12 models, accuracy falls from 39.66% on low-difficulty tasks to 23.50% on medium and 16.76% on high. GPT-6 Astra alone drops from 59.26% to 40.98% to 24.71%. On the hardest tier, Gemini 3.8 Flash leads at 28.24% while Qwen3.8-Flash manages only 4.71%.\n\n## Humans at 21.1%, below the top model\n\nAcross 1,000 independent solving attempts, human participants reached 21.1% overall accuracy, below the best-performing model. And for 42.5% of the tasks, **no assigned human participant solved them at all.**\n\nThat is not the same as saying models have surpassed human researchers — the tool conditions and time budgets on both sides may not be comparable. But it does show that when a question demands multi-hop evidence gathering across multiple sources and modalities, humans fail at scale too.\n\n## More tokens does not mean more accuracy\n\nGPT-6 Astra averaged 51.05 tool calls and 18.66 minutes per task. GPT-5.6 Sol made nearly the same number of calls (52.42) but finished in 8.99 minutes, scoring 11 points lower. Gemini 3.8 Flash consumed roughly 5.22 times GPT-6 Astra's tokens and scored lower still. Tool composition explains just as little: Claude Opus 5.5 and Qwen3.8-Flash spend near-identical shares of calls on GUI work (51.52% versus 49.42%), yet they score 38.0% and 12.0%.\n\n## Failure is what actually costs money\n\n**Failed tool calls account for only 2.83%–9.42% of all calls, yet consume 19.27%–61.90% of total tool execution time.** For GPT-6 Astra the split is 6.34% of calls against 42.57% of the time.\n\nThe budget is a one-hour wall-clock limit plus a 100-call limit. Exhaustion ranges from 16.5% for Claude Opus 5.5 to 81.5% for Qwen3.8-Flash. After a failure, Claude, Gemini, and GPT-5.6 Sol switch to a different tool roughly 79%–83% of the time, while Kimi K3 and GLM-5.3 retry with modified arguments about 37%–38% of the time. One group changes lanes; the other spins in place.\n\n## So what\n\nThe value of this evaluation ([arXiv:2610.30181](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30181)) is not the 39.0%. It is that it takes \"an agent doing deep research\" apart into three measurable things: **picking the right sources, coordinating different tools, and stopping before the budget runs out.**\n\nThe first two set the ceiling. The third sets the cost curve. An agent that averages 18 minutes and 50 tool calls, and one that runs 45 minutes without an answer, should not be priced as the same product.\n\nThe benchmark does not settle whether AI can do deep research for us. It leaves a sharper question: when your agent hits a dead end, is it changing tools, or just editing the same arguments?\n\n([arXiv:2610.30181](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30181))","opends-world-benchmark-deep-search-agents","2026-10-23T11:20:00Z","2026-10-23T11:07:16.302796Z","2026-10-23T11:07:16.302804Z",true,"agent",35,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"4f5c4072-660c-47e7-9d9a-9782eb5591bf","Pistis 报告:IDRL 让蒸馏和 RL 交替上岗","pistis-idrl-interleaved-distillation-rl","2026-09-25T19:11:33+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"6ca2aad2-fc68-46ab-8120-3eada9c187e2","模型自己操控相机建图:最强 VLM 平均 IoU 只有四成","spatialscout-vlm-active-3d-mapping-benchmark","2026-10-23T03:30:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"ace7a537-e2de-448f-811f-a028357aed42","FocusGUI:给全模态GUI智能体装上注意力滤镜","focusgui-omni-modal-gui-agents","2026-10-21T19:05:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"aa718f3c-14cf-4984-8762-794eaaee8c16","MAI-Gym:9B 智能体在 2 万个并行环境里做在线 RL","mai-gym-gui-agent-online-rl-9b","2026-10-21T09:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"15da3422-7db5-4da6-a12a-e4fd84b9a50b","视觉脚手架:88M 参数把视觉特征搬进思维链","visual-scaffolding-native-multimodal-reasoning","2026-10-20T23:03:57+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"2c7a8ca0-ab46-454b-ae16-aa7e7b413753","Ultramarine 1 Flash:不可验证任务RL","ultramarine-1-flash-non-verifiable-rl","2026-10-20T07:07:00+00:00"]