[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-histeve-hierarchical-vla-minecraft-agent":3,"topics-all":38,"news-related-db8f2b94-8565-49fe-9fd8-08310f441161":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"db8f2b94-8565-49fe-9fd8-08310f441161","HiSTEVE 1.0:像人一样玩我的世界,只靠屏幕和键鼠","分层VLA智能体HiSTEVE 1.0只用第一人称画面和原生键鼠玩《我的世界》:GPT-4o挑子任务,Qwen2.5-VL-7B执行,专训完成检测器F1 0.863;论文自报MineStudio Bench 85.0%、OpenHA 37.4%,并从零合成铁镐,GUI与战斗仍略逊基线。","在《我的世界》里从零合成一把铁镐，意味着挖矿、烧炼、合成的多阶段连续作业——过去能做到这件事的 AI，要么眼睛长在环境接口上，要么手长在脚本里。10 月 22 日提交到 arXiv 的 HiSTEVE 1.0 给出了第三种答案：只看第一人称画面、只动原生键鼠，像人类玩家一样把整条链路跑通。HF 论文页将团队标注为 Tencent。\n\n## 两派 agent 的老断层\n\n《我的世界》agent 研究长期分两派：VPT、STEVE-1 一脉有原生视觉运动控制，却没有长程自主性；Voyager 式高层规划靠环境 API 和预定义技能执行，离开特权接口就失能。HiSTEVE 的切入点是把两边缝起来——不读游戏内部状态、不调环境 API，规划与执行分层协同，这套组合也被论文称为能力感知的 agent harness。\n\n## 分层设计：规划器、检测器、执行器\n\n编排层带一个结构化记忆负责追踪任务进度，GPT-4o 作为规划器在 VLA 能力圈内挑选下一个原子子任务。执行层是单一 VLA，从 Qwen2.5-VL-7B 初始化，纯视觉输入，以 20 Hz 输出键鼠动作，覆盖全部子任务。\n\n分层系统最难的是「子任务什么时候算完」。团队为此专门训练了一个策略耦合的完成状态检测器：Qwen2.5-VL-3B 底座，共享 VLA 的图像编码器，再读取 800\u002F600\u002F400\u002F200 毫秒间隔的历史帧。在 4,674 个 VPT 轨迹样本的验证集上，这个论文表格标注为 1.4B 的检测器准确率与 F1 双双达到 0.863，高于被提示词驱动的 GPT-5.5（0.776）；Qwen3.5-35B 的「未完成」类召回只有 0.042，基本判不动。专用小模型在窄任务上赢过通用大模型，这里是一个干净的例证。\n\n## 两个容易被忽略的工程细节\n\nGUI 与非 GUI 交互共享全部参数时，优化信号会互相打架。团队的对策一是 regime-specific RMSNorm：每层给两类交互各配一个独立缩放向量，额外参数只有层数乘隐藏维度两倍的量级，开销可忽略；二是给鼠标移动单独一套 token 嵌入。训练分三阶段推进：世界知识注入、模仿学习、带环境反馈的强化学习，逐段提升可靠性与失败恢复。\n\n## 成绩单与冷水\n\n按论文自报的对比，HiSTEVE 在 MineStudio Bench（20 个原子任务、每任务三次尝试）拿到 85.0% 的严格 pass@1，对比里最强的基线 OpenHA Text\u002FCoA 为 65.0%；OpenHA 平均成功率 37.4% 对 36.8%，优势集中在 Embodied 家族（37.0% 提到 46.0%）；长程 LongBench（14 个只给目标的任务、49 个隐藏诊断节点）里 Resource & Crafting 家族成功率达 42.9%，比基线高 28.6 个百分点。\n\n但论文自己也写得很清楚：GUI 与 Combat 两族略低于基线——失败案例能摸到正确界面，却断在多步物品放置、中间态纠正、结果取回这些环节；Recovery 与 Open-ended 任务仍然很难。另外，规划器用的是闭源的 GPT-4o，整套系统的「自主」有一段握在别人的 API 里。\n\n## 游戏为什么值得认真看\n\n游戏是具身智能最便宜的压力测试场：规则开放、反馈即时、失败零成本。更重要的是，human-interface 路线——不靠特权接口、只用屏幕和键鼠——恰好是真实世界 GUI agent 的镜像问题：让 AI 操作一台普通电脑，和让它玩一个开放世界游戏，考验的是同一种能力。等这条路线成熟，最先被重新定义的不是游戏，而是「坐在电脑前的工作」本身（论文全文见 [arXiv:2610.31162](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.31162)）。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.31162","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"5185730c-136c-4896-b4ef-5d8f96035c54","en","HiSTEVE 1.0: Hierarchical VLA Plays Minecraft Like a Human","GPT-4o-planned VLA plays Minecraft with vision-only keyboard-mouse. 85.0% MineStudio Bench, iron pickaxe from scratch; GUI\u002FCombat below baselines.","Crafting an iron pickaxe from scratch in Minecraft is a multi-stage pipeline — mine, smelt, craft — and until now the AIs that managed it either saw the world through environment APIs or had their hands scripted. HiSTEVE 1.0, submitted to arXiv on October 22, offers a third answer: first-person pixels only, native keyboard-and-mouse only, the whole chain run the way a human player would. The Hugging Face paper page badges the team as Tencent.\n\n## The old gap between two agent camps\n\nMinecraft agent research has long split in two. The VPT and STEVE-1 lineage has native visuomotor control but no long-horizon autonomy; Voyager-style planners run high-level plans through environment APIs and predefined skills, and go limp without privileged interfaces. HiSTEVE stitches the two together — it reads no internal game state and calls no environment API, pairing hierarchical planning with native execution in what the paper calls a capability-aware agent harness.\n\n## The hierarchy: planner, detector, executor\n\nAt the orchestration level, a structured memory tracks task progress while GPT-4o acts as the planner, picking the next atomic subtask from within the VLA's capabilities. At the control level, a single VLA initialized from Qwen2.5-VL-7B takes vision-only input and emits keyboard-and-mouse actions at 20 Hz across all subtasks.\n\nThe hardest part of any hierarchical system is knowing when a subtask is done. The team trained a policy-coupled task-completion detector: a Qwen2.5-VL-3B backbone sharing the VLA's image encoder, reading historical frames sampled at 800\u002F600\u002F400\u002F200 ms. On a class-balanced validation split of 4,674 VPT-trajectory samples, this detector — listed at 1.4B in the paper's table — reaches 0.863 in both accuracy and F1, ahead of prompted GPT-5.5 at 0.776; Qwen3.5-35B manages only 0.042 recall on the incomplete class. A purpose-built small model beating a general-purpose large one on a narrow job — a clean example.\n\n## Two engineering details easy to miss\n\nWhen GUI and non-GUI interactions share every parameter, their optimization signals fight. The fixes: regime-specific RMSNorm, giving each interaction regime its own per-layer scale vector at a parameter cost of roughly two-times-layers-times-hidden-dim — negligible; and a separate token embedding for mouse movement. Training proceeds in three stages: world-knowledge injection, imitation learning, then reinforcement learning with environment feedback.\n\n## The scoreboard, and the cold water\n\nIn the paper's own comparison, HiSTEVE scores 85.0% strict pass@1 on MineStudio Bench (20 atomic tasks, three trials each) against 65.0% for the strongest baseline, OpenHA Text\u002FCoA. On OpenHA it averages 37.4% versus 36.8%, with the gain concentrated in the Embodied family (37.0% up to 46.0%). On the long-horizon LongBench (14 goal-only tasks, 49 hidden diagnostic nodes), the Resource & Crafting family reaches 42.9%, a 28.6-point jump over the baseline.\n\nBut the paper is blunt about the weak flanks: GUI and Combat families sit slightly below the baseline — failed GUI episodes reach the right interface, then break on multi-step item placement, intermediate-state correction, or result retrieval; Recovery and Open-ended tasks remain hard. And since the planner is closed-source GPT-4o, a stretch of the system's \"autonomy\" lives in someone else's API.\n\n## Why games deserve a serious look\n\nGames are the cheapest stress-test arena for embodied intelligence: open rules, instant feedback, zero-cost failure. More to the point, the human-interface route — no privileged API, just screen and keyboard — is the mirror problem of real-world GUI agents: making an AI operate an ordinary computer and making it play an open-world game test the same capability. When this route matures, the first thing redefined will not be games, but \"work that happens at a computer\" itself (full paper: [arXiv:2610.31162](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.31162)).\n","histeve-hierarchical-vla-minecraft-agent","2026-10-26T15:13:44Z","2026-10-26T15:13:52.995411Z","2026-10-26T15:13:52.995421Z",true,"agent",209,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"6ca2aad2-fc68-46ab-8120-3eada9c187e2","模型自己操控相机建图:最强 VLM 平均 IoU 只有四成","spatialscout-vlm-active-3d-mapping-benchmark","2026-10-23T03:30:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"9a66407a-b79a-4a12-a4ef-b0d7018c8415","字节Seed新论文:VLM操作3D编辑器摆家具,把真实房间变成仿真场景","lucida-vlm-gizmoact-real-to-sim","2026-09-01T19:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"fd0e593c-bb2c-44f9-8913-16f8dcef6999","苹果8B智能体:真实iOS上追平Gemini-3.1-Pro","apple-online-rl-ui-agent-live-ios","2026-10-31T11:10:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"9109e6b0-bf85-472d-abf6-ba8e31dc17c5","Co-Design 基准:七成准则过,26.4% 完整交付","co-design-bench-agent-delivery-gap","2026-10-30T05:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"1d85eeac-c803-453e-99f2-cad5fa838d1e","AMBIT:空间音频的编辑接口,是一份可执行的场景计划","ambit-spatial-audio-scene-plan","2026-10-27T09:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"e4ac848d-d24a-465f-bece-37e090da86de","全双工语音模型长对话必崩:KAIST 揪出初始化锚点被挤走","full-duplex-speech-collapse-fd-lhb-kv-cache","2026-10-27T03:20:00+00:00"]