[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-spatialscout-vlm-active-3d-mapping-benchmark":3,"topics-all":38,"news-related-6ca2aad2-fc68-46ab-8120-3eada9c187e2":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"6ca2aad2-fc68-46ab-8120-3eada9c187e2","模型自己操控相机建图:最强 VLM 平均 IoU 只有四成","北大团队的 SpatialScout 把空间智能评测从选择题改成实操:80 个场景、3140 个目标物体,模型自己操控相机探索建图。七个前沿模型中,论文口径最强的 Claude Opus 5.5 全目标平均 IoU 仅 41.71%,IoU 超过 0.9 的目标只有 3.25%。","过去几年关于「空间智能」的评测基本都在做同一件事:给模型一张图或一段视频,问它「沙发在电视左边还是右边」,答对就算赢。这类任务只能证明模型认得出物体,证明不了它能把物体放进一张带尺度的三维地图里。论文 arXiv:2610.26659(https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.26659) 换了个考法:不给选择题,让模型自己拿着相机去探索房间。\n\n## 任务设计:模型自己拿相机\n\n基准由北京大学团队构建,Zhaoxi Zhang 担任项目负责人、Weikang Li 为通讯作者,联合天津大学,许可为 CC BY 4.0,代码开源在 https:\u002F\u002Fgithub.com\u002FJesseZZZZZ\u002FSpatialScout 。\n\n场景部分包含 80 个室内场景、共 3140 个目标物体,每个场景 30 到 40 个目标,并且刻意塞进了同类别的重复实例。模型的起点是一张 RGB 图、一份物体清单(实例编号、类别、等价组),以及相机内参与位姿。之后它要靠十个共享工具自己干活:list_entities 查清单,get_camera_pose 和 get_camera_intrinsics 读取标定,set_camera_position、move_camera、rotate_camera 调整视角,capture_image 抓帧,triangulate_observations 做多视角三角化,submit_positions 提交或修订物体中心,finish_mapping 收工。没有任何一个工具会泄露真值位置或评分。\n\n预算卡得很紧:每个 episode 允许 100 次模型调用、100 张图像(含初始那张),用满之后剩余调用只能用来推理和提交。渲染出的图像是 512×512 的 RGB,交付时压成最长边不超过 384 像素的 JPEG,模型需要自己把坐标换算回原始像素网格。真值中心的定义也很朴素——渲染几何体的紧致世界轴对齐包围盒的中点,带关节的物体取各连杆包围盒的并集。\n\n## 跑分:论文里最强的也只有四成\n\n论文在统一的 80 个场景上跑了七个前沿模型,以下是全目标(object-micro)的 IoU 统计,均为论文官方口径:\n\n- **Claude Opus 5.5**(xhigh):平均 IoU 41.71,正重叠召回 77.36,IoU 超过 0.5 的目标 47.23%,超过 0.9 的只有 3.25%\n- **Claude Sonnet 5.5**:平均 32.26,IoU>0.5 为 33.60,IoU>0.9 为 1.40\n- **GPT-6 Astra**(xhigh):平均 33.16,IoU>0.5 为 35.15,IoU>0.9 为 1.62;切到 medium 推理档掉到 31.18\n- **GPT-6 Sol**:平均 20.83,IoU>0.5 为 18.95,IoU>0.9 为 0.48\n- **Qwen3.8 Max**:平均 6.80,IoU>0.5 为 4.52,IoU>0.9 为 0.00\n- **Kimi-K3**:平均 4.22,IoU>0.5 为 2.77,IoU>0.9 为 0.00\n- **DeepSeek V4.1 Flash**:平均 3.48,IoU>0.5 为 0.92,IoU>0.9 为 0.00\n\n如果只在「已经提交的目标」上算,Claude Opus 5.5 的平均 IoU 升到 46.69,GPT-6 Astra 是 41.16。这段差额就是漏报吃掉的分量。\n\n## 工具调用把行为差异照了出来\n\n论文的工具调用频率图给出了比分数更有意思的线索。Claude Opus 5.5 打的工具调用最多,论文认为它的广泛探索换来了更好的地图质量;GPT-6 Astra 用明显更少的调用拿到了接近的成绩。相反,DeepSeek V4.1 Flash 和 Qwen3.8 Max 大量轮次花在移动相机上,观察和建图的调用偏少——论文的解读是,这类模型难以挑选有信息量的视角,也不善于把视觉观察转成连贯地图;而且无效视角还要吃掉上下文,图像本身占的上下文长度并不小。\n\n## 失败的主要形态是漏,不是偏\n\n论文分析了四个模型在同一批十个任务上的 40 条轨迹。判定重建成功需要同时满足两个条件:物体覆盖率不低于 90%,且匹配上的已提交目标平均中心误差不超过 0.2 米。最终只有 6 条轨迹达标,而这 6 条的共同点是都有充分的多视角几何支撑、增量提交持续扩大覆盖、以及后续的持续修正。\n\n34 条失败轨迹里,覆盖不完整是最主要的形态,出现在 26 条;其中 21 条出现「三角化残差很低但位置仍然明显不准」的情况,论文把这类问题指向视觉对应关系错误或物体中心选错——纯几何一致性无法解决。还有 13 条带着几何测量不一致的警告,另有 13 条在至少还剩十轮响应额度时就提前结束。\n\n## 所以呢\n\n这份基准刺破的是「视觉常识即空间能力」这个错觉。识别出一个物体、把左右前后说对,和把它的中心放进共享坐标系里精确到厘米,是两种不同难度的能力;前者靠图文对齐的统计规律就能蒙对一部分,后者要求跨视角关联、几何估计和持续修正同时在线。\n\n有意思的是,基准其实给了一个三角化工具,能在至少两个标定过的视角上算出三维点,并回报垂直线残差和最大视线夹角。但论文明确写道:低残差本身既不能证明标记的像素指向同一个物体特征,也不能证明算出的点和物体中心的关系——几何工具把「算得准不准」交出去,却完全不负责「算的是不是同一个东西」。这大概是当前具身智能落地时最真实的卡点:传感器和几何管线可以很精确,但没人替模型判断它到底在看哪个物体。\n\n论文在 2026 年 10 月 22 日的 Hugging Face Daily Papers 日榜上出现过,想复现的可以直接拉那个开源仓库。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.26659","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"66a2f6de-cdc6-41b5-ad03-ae95ca752cd8","en","Frontier VLMs Reach Only 41.71 IoU At Active 3D Mapping","SpatialScout replaces multiple-choice spatial eval with hands-on 3D mapping: the agent drives the camera. Claude Opus 5.5 leads at 41.71 mean IoU.","For the last few years, most evaluations of \"spatial intelligence\" have done the same thing: hand the model a single image or a short video clip, ask whether the sofa is to the left or right of the television, and declare victory on a correct answer. That format can prove a model recognizes objects. It cannot prove the model can place those objects into a metric 3D map. The paper arXiv:2610.26659 changes the exam format: no multiple choice. The model gets the camera and has to go explore the room on its own.\n\n## Task design: the model holds the camera\n\nThe benchmark was built by a Peking University team, with Zhaoxi Zhang as project lead and Weikang Li as corresponding author, jointly with Tianjin University. It is released under CC BY 4.0, and the code is open at https:\u002F\u002Fgithub.com\u002FJesseZZZZZ\u002FSpatialScout .\n\nThe scene set comprises 80 indoor scenes containing 3,140 target objects, with 30 to 40 targets per scene, spanning diverse spatial layouts, object scales, and repeated category instances. The agent starts with one RGB image, an object inventory (instance identifier, category, equivalence group), plus camera intrinsics and poses. From there it works through ten shared tools: list_entities to read the inventory, get_camera_pose and get_camera_intrinsics for calibration, set_camera_position, move_camera and rotate_camera to steer, capture_image to grab frames, triangulate_observations for multi-view triangulation, submit_positions to write or revise object centers, and finish_mapping to stop. No tool reveals ground-truth target positions or evaluation scores.\n\nThe budget is tight. Each episode allows 100 model calls and 100 acquired images, including the initial one; once captures are exhausted, the remaining calls can only be used for reasoning and submission. Explicit captures render 512x512 RGB images, delivered as JPEGs with the longest side at most 384 pixels, so the agent has to convert delivered coordinates back onto the original image grid. Ground-truth centers are defined simply: the midpoint of the tight world-axis-aligned bounding box of the rendered geometry, with the union of link boxes for articulated objects.\n\n## Scores: the strongest model in the paper reaches only the low forties\n\nThe paper evaluates seven frontier models on the shared 80-scene set. All numbers below are the paper's official figures, object-micro over all targets:\n\n- **Claude Opus 5.5** (xhigh): mean IoU 41.71, positive-overlap recall 77.36, 47.23% of targets above IoU 0.5, and only 3.25% above 0.9\n- **Claude Sonnet 5.5**: mean 32.26, IoU>0.5 at 33.60, IoU>0.9 at 1.40\n- **GPT-6 Astra** (xhigh): mean 33.16, IoU>0.5 at 35.15, IoU>0.9 at 1.62; dropping to the medium reasoning setting pulls it down to 31.18\n- **GPT-6 Sol**: mean 20.83, IoU>0.5 at 18.95, IoU>0.9 at 0.48\n- **Qwen3.8 Max**: mean 6.80, IoU>0.5 at 4.52, IoU>0.9 at 0.00\n- **Kimi-K3**: mean 4.22, IoU>0.5 at 2.77, IoU>0.9 at 0.00\n- **DeepSeek V4.1 Flash**: mean 3.48, IoU>0.5 at 0.92, IoU>0.9 at 0.00\n\nRestricting the average to objects the models actually submitted lifts Claude Opus 5.5 to 46.69 and GPT-6 Astra to 41.16. That gap is exactly what omissions cost each model.\n\n## Tool-call patterns expose behavioral differences\n\nThe paper's tool-call frequency figures offer a more revealing signal than the scores alone. Claude Opus 5.5 makes the largest number of tool calls, and the authors read its broad exploration as the reason for better map quality. GPT-6 Astra reaches competitive mapping performance with noticeably fewer calls. In the opposite direction, DeepSeek V4.1 Flash and Qwen3.8 Max spend many turns moving the camera while observing and mapping far less. The paper interprets this as difficulty selecting informative viewpoints and difficulty converting visual observations into a coherent map. Uninformative views are not free either, since images consume substantial context length.\n\n## The dominant failure mode is omission, not offset\n\nThe paper analyzes 40 trajectories from four models on the same ten selected tasks. Reconstruction success requires both at least 90% object coverage and a mean center error of at most 0.2 meters over matched submitted objects. Only six trajectories clear both bars, and all six share the same profile: extensive multi-view geometric grounding, incremental submission that expands coverage, and subsequent refinement.\n\nAmong the 34 unsuccessful trajectories, incomplete coverage is the most common pattern, appearing in 26 of them. In 21 cases at least one submitted position stays substantially inaccurate despite a low triangulation residual, which the authors attribute to incorrect visual correspondences or wrong object-center selection, neither of which geometric consistency alone can resolve. Thirteen cases retain inaccurate positions alongside geometric-inconsistency warnings, and another overlapping set of thirteen terminates with at least ten response rounds still unused.\n\n## So what\n\nThe benchmark punctures the comfortable assumption that visual common sense equals spatial competence. Recognizing an object and getting left\u002Fright\u002Ffront\u002Fback right is one kind of ability; putting its center into a shared coordinate frame at centimeter precision is another. The first can be partly guessed from statistical regularities in image-text alignment. The second demands cross-view association, geometric estimation and continuous revision all working at once.\n\nWhat makes this interesting for practitioners is that the benchmark does hand the model a triangulation tool that estimates a 3D point from correspondences across at least two calibrated views, and reports the root-mean-square perpendicular line residual along with the maximum pairwise ray angle. The paper is explicit that a low residual establishes neither whether the marked pixels identify the same object feature nor how the estimated point relates to the object's center. The geometry pipeline can be very precise, yet nothing in that pipeline decides whether the model is even looking at the right thing. That is a fair description of the hardest blocker in embodied AI today.\n\nThe paper appeared on the Hugging Face Daily Papers board for October 22, 2026, and the repository is public for anyone who wants to reproduce the runs.","spatialscout-vlm-active-3d-mapping-benchmark","2026-10-23T03:30:00Z","2026-10-23T03:11:21.237406Z","2026-10-23T03:11:21.237416Z",true,"agent",13,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"9a66407a-b79a-4a12-a4ef-b0d7018c8415","字节Seed新论文:VLM操作3D编辑器摆家具,把真实房间变成仿真场景","lucida-vlm-gizmoact-real-to-sim","2026-09-01T19:10:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"6b2cf1d1-7131-4885-a049-730b82846961","RISE-Agent 拆开视觉编辑推理盲点:58 个模型里,最强闭源只 56.6%","rise-agent-visual-editing-benchmark","2026-10-13T11:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"c368ad9f-9308-4a0b-8f5c-3ae4601b48b9","D2K-Bench: 专家设计把 LLM 写 GPU 核提速 33.9%","d2k-bench-llm-gpu-kernel-design-guidance","2026-10-07T03:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"fe5546f3-09e4-42a3-bfde-6bbd0e8d474c","外星世界实测:探索 4 轮 87.6%,垫底 12.9%","explorationbench-alien-worlds","2026-09-28T21:07:30+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"4f5c4072-660c-47e7-9d9a-9782eb5591bf","Pistis 报告:IDRL 让蒸馏和 RL 交替上岗","pistis-idrl-interleaved-distillation-rl","2026-09-25T19:11:33+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"051084dc-3da0-451e-9c6b-a267d5b0e77f","给机器人技能装上门禁:EmbodiedSkills 预检+验证闭环,RoboTwin 50 任务冲到 86.2%","embodiedskills-vla-verify-loop","2026-09-08T17:10:00+00:00"]