[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-microsoft-mage-vl-cowork-hybrid-computer-use":3,"topics-all":38,"news-related-26337a90-e049-44cb-9738-16d10a1fe84b":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"26337a90-e049-44cb-9738-16d10a1fe84b","微软4B混合智能体:OSWorld 66%,点击还是敲代码","微软开源 4B 混合计算机操作智能体 Mage-VL-Cowork:一个策略内部同时调度 GUI 点击与终端代码执行,论文自测在 OSWorld-Verified 上拿到 66.0%,压过 30B、72B 等更大开源智能体。关键不在规模,而在先把教师 harness 剪干净再蒸馏。","现在几乎所有能操作电脑的大模型都在做同一件事:看截图、算坐标、点一下。微软 Mage 团队开源的 Mage-VL-Cowork 抛出的问题很直接——一只靠像素的智能体要交多少税?一张 1920×1080 的截图就是几千 token,一次结构化的表格改动要靠几十次鼠标拖拽重新推导一遍,而这台机器上明明有一条更快、更可验证的通道:直接写代码。\n\n这个 4B 参数的模型由 Mage-ViT(16×16 patch、3D RoPE)加 Qwen3-4B 组成,权重以 Apache-2.0 发布在 [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FMage-VL-Cowork),完整技术报告见 [arXiv](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40747)。它把 GUI 与终端收进同一个策略内部的两条动作通道,没有外接编排器,每一步自己决定该点还是该敲。\n\n## 分数:66.0%,以及它说明了什么\n\n最终 RL checkpoint 在 OSWorld-Verified 上拿到 66.0%(361 道保留任务,排除 Google Drive,50 步预算,两次评测平均 65.33% 与 66.75%)。分项是 OS 87.5、Office 64.9、Daily 65.0、Professional 66.3、Workflow 62.7。\n\n论文自己跑的对比表里,4B 档位另一款 Fara1.5-4B 只有 13.9%;往上,30B 的 Muse Glimmer-30B 是 65.9%,27B 的 Qwen3.6-27B 是 63.9%,32B 的 EvoCUA-32B 是 56.7%,72B 的 OpenCUA-72B 是 44.9%。开源阵营里压过它的只有 9B 的 ScaleCUA-Qwen3.5-9B(68.7%)。闭源那边,人类参考线 72.4%,GPT-5.5 是 78.7%,GPT-5.4-mini 是 69.6%。\n\n论文报告的\"同参数量级最优\"具体指:4B 这一档没有对手,不是全面领先。压过 30B 靠的是 0.1 个百分点,压过 72B 靠的是 21 个百分点。要留个心眼的是,这张表由论文作者自己跑,同行的分数也都出自这张表,不是各家独立复测的结果。\n\n## 真正值钱的不是分数,是\"先改教师\"\n\n最能说明问题的是一组 harness 对照。教师模型固定为 GPT-5.4,在 672 道跨应用保留任务上,只保留 GUI + code 的精简 harness 教师拿到 92.0%、平均 25.0 步;加一条 accessibility 通道的版本反而只有 89.6%、35.3 步。用各自教师的轨迹去训 4B 学生,差距被放大成 37.6% 对 25.1%——12.5 个百分点,轨迹还短了 21.5%。\n\n工具越多不等于越强。前沿教师能靠多走几步消化掉冗余通道,4B 学生却对被切碎的监督极其敏感。\n\n## 数据合成:每个任务成本砍到六分之一\n\nMage-CUA-Forge 不再为每个任务跑一次开放式 agent 循环,而是每个原子技能写一次可复用的工作流。在同一套生成模型下,每任务 token 从 1,276K 降到 85K(-93.3%),成本从 0.906 美元降到 0.158 美元(-82.6%)。经过三轮学生反馈迭代,流水线合成 293,806 个任务、覆盖 6,915 个原子技能,其中 38.24% 跨多个应用;最终保留 318,837 条混合轨迹,GUI 调用占 68.19%,终端占 31.81%。\n\n## 五阶段课程与在线 RL\n\n路径是:GUI 定向中期训练 → GUI Agent SFT → 代码与终端 SFT → 混合 CUA SFT → 在线混合 RL(GRPO、异步 rollout、可验证任务奖励)。先把两条通道各自练硬,再教它什么时候切换,而不是让路由成为 GUI 能力薄弱的临时补丁。\n\n分数沿着这条路走:45.56% → 55.84% → 66.0%。第一阶段把 OSWorld-G 从 30.6% 拉到 56.9%,ScreenSpot-v2 从 78.8% 拉到 87.2%,双双越过 Qwen3.5-4B(54.5%)和 Qwen3-VL-4B(85.3%)。\n\n在线 RL 那一段最反常识:成绩涨了,消耗反而降了——平均模型调用 21.57 → 18.77(-13.0%),每任务输出 token 7,374 → 5,499(-25.4%)。对照 GPT-5.4-mini 的 18.24 次调用与 12,397 个输出 token,4B 模型在轮次上基本持平,输出 token 少了 55.6%,论文称之为 2.25 倍 token 效率。\n\n## 我的看法\n\n这条路线里最值得抄的不是架构,是\"先把教师 harness 改对,再去收数据\"这个顺序。过去两年谈智能体,默认模型是瓶颈;这篇把镜头反了过来——教师脚手架上多一条冗余工具,4B 学生就掉 12.5 分,而这种损失在训练日志里几乎看不见。\n\n另一个方向是\"混合\"正在成为默认形态。纯 GUI 路线撞的是 token 墙和可靠性墙,纯代码路线看不见画布,混合策略把选择权交还给模型本身。官方[项目页](https:\u002F\u002Fmicrosoft.github.io\u002FMage\u002Fcowork\u002F)给的一段实时对拍很说明问题:同一个 LibreOffice Calc 合并单元格、加图表的任务,4B 模型用 Python 在终端改表,39 秒、6 步跑完;27B 的 Qwen3.6-27B 纯靠鼠标拖拽,9 分 56 秒、39 步。这是单个演示,不能当普遍结论,但方向清楚。\n\n至于\"端侧\"这层:独立站点 modelfit.io 的统计显示,这个模型目前唯一的量化构建是 2.716 GB 的 Q4_K_M GGUF 外加 0.353 GB 视觉投影器,16GB 内存的 Mac 装得下。但它要驱动的是一整台 Linux 桌面,离\"手机上的智能体\"还有距离。\n\n所以真正该问的不是\"小模型能不能追上大模型\",而是:当智能体的能力差距越来越不由参数量决定的时候,它到底由谁写的脚手架决定?","https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FMage-VL-Cowork","24d5c6c5-6573-4180-a1fd-f1459842d1af",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"5a874765-ede2-4eb3-9806-3f34e4c8b565","en","Microsoft's 4B Hybrid Computer-Use Agent: 66% on OSWorld","Microsoft open-sources Mage-VL-Cowork, a 4B hybrid agent routing GUI clicks and terminal code, scoring 66.0% on OSWorld and beating 30B and 72B models.","Almost every model that drives a computer does the same thing: look at a screenshot, compute a coordinate, click. Microsoft's Mage team open-sourced a model that asks a blunt question — how much tax does a pixel-only agent pay? A single 1920×1080 screenshot burns thousands of tokens, and re-deriving one structured spreadsheet edit through dozens of mouse drags throws away the fastest, most verifiable channel on the machine: writing code.\n\n[Mage-VL-Cowork](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FMage-VL-Cowork) is that bet. A 4B-parameter model built on Mage-ViT (16×16 patch grid, 3D RoPE) plus Qwen3-4B, released under Apache-2.0 with the full [technical report on arXiv](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40747). It folds GUI and terminal into two action channels inside a single policy — no external orchestrator — and decides at every turn whether to click or to type.\n\n## The score: 66.0%, and what it actually means\n\nThe final RL checkpoint reaches **66.0% on OSWorld-Verified** — 361 held-out tasks, Google Drive excluded, 50-step budget, averaged over two evaluation runs (65.33% and 66.75%). Category breakdown: OS 87.5, Office 64.9, Daily 65.0, Professional 66.3, Workflow 62.7.\n\nIn the paper's own comparison table, the other 4B entry, Fara1.5-4B, manages 13.9%. Going up in size, Muse Glimmer-30B scores 65.9%, Qwen3.6-27B 63.9%, EvoCUA-32B 56.7%, OpenCUA-72B 44.9%. Among open models only ScaleCUA-Qwen3.5-9B (68.7%) edges past it. On the closed side, the human reference line sits at 72.4%, GPT-5.5 at 78.7%, and GPT-5.4-mini at 69.6%.\n\nThe paper's \"state-of-the-art at its parameter size\" claim therefore means one specific thing: nobody else occupies the 4B tier, not that it leads everywhere. Beating the 30B model costs 0.1 percentage point; beating the 72B model costs 21. One caveat worth keeping: the table was run by the paper's own authors, and the competing scores all come from that same table rather than independent replications.\n\n## The valuable part is not the score, it is \"fix the teacher first\"\n\nThe clearest evidence is a harness ablation. With the teacher model pinned to GPT-5.4, on 672 held-out multi-application tasks, a pruned GUI-plus-code harness teacher scores 92.0% at 25.0 mean steps; the variant with an extra accessibility channel drops to 89.6% at 35.3 steps. Train 4B students on each teacher's trajectories and the gap widens to 37.6% versus 25.1% — a **12.5 percentage-point** swing, with trajectories 21.5% shorter.\n\nMore tools does not mean a stronger agent. A frontier teacher can absorb a redundant channel by taking extra steps; a 4B student is acutely sensitive to supervision that has been split apart.\n\n## Data synthesis: per-task cost cut to a sixth\n\nMage-CUA-Forge stops running an open-ended agent loop per task and instead authors one reusable, skill-conditioned workflow per atomic skill. Under the same generation model, tokens per task drop from 1,276K to 85K (−93.3%) and cost falls from 0.906 USD to 0.158 USD (−82.6%). After three rounds of student-feedback iteration the pipeline has synthesized **293,806 tasks across 6,915 atomic skills**, 38.24% of them spanning multiple applications, and retains **318,837 verified hybrid trajectories** — 68.19% GUI tool calls, 31.81% terminal calls.\n\n## Five stages, and an RL stage that gets cheaper as it gets better\n\nThe recipe runs: GUI-oriented midtraining → GUI agent SFT → coding and terminal SFT → hybrid CUA SFT → online hybrid RL (asynchronous GRPO rollouts with verifiable task rewards). Build each channel on its own first, then teach the model when to switch, rather than letting routing become a patch over weak GUI execution.\n\nThe score climbs along that path: 45.56% → 55.84% → 66.0%. Stage 1 alone takes OSWorld-G from 30.6% to 56.9% and ScreenSpot-v2 from 78.8% to 87.2%, passing Qwen3.5-4B (54.5%) and Qwen3-VL-4B (85.3%).\n\nThe RL stage is the counterintuitive part: accuracy rises while consumption falls. Mean model calls per task go 21.57 → 18.77 (−13.0%), mean output tokens 7,374 → 5,499 (−25.4%). Against GPT-5.4-mini — 18.24 calls and 12,397 output tokens — the 4B model matches it on interaction turns while emitting 55.6% fewer output tokens, which the team puts at 2.25× more token-efficient.\n\n## What I think\n\nThe part of this work worth copying is not the architecture. It is the ordering: fix the teacher's harness before you collect data. Two years of agent talk has assumed the model is the bottleneck. This paper turns the camera around — one redundant tool bolted onto the teacher scaffold costs a 4B student 12.5 points, and that loss is essentially invisible in the training logs.\n\nThe second layer is that \"hybrid\" is becoming the default shape. A pure-GUI route runs into the token wall and a reliability wall; a pure-code route cannot see the canvas. A hybrid policy hands the choice back to the model. The side-by-side demo on the [project page](https:\u002F\u002Fmicrosoft.github.io\u002FMage\u002Fcowork\u002F) shows the shape clearly: on the same LibreOffice Calc task — merge banner ranges, label them, add a chart — the 4B model rewrites the workbook with Python in the terminal and finishes in 39 seconds across 6 steps, while Qwen3.6-27B clicks and drags for 9 minutes 56 seconds across 39 steps. One demo is not a general claim, but the direction is clear.\n\nOn \"on-device,\" the honest version: independent tracker ModelFit lists the only quantized build as a 2.716 GB Q4_K_M GGUF plus a 0.353 GB vision projector, so it fits on a 16GB Mac. But this thing drives a full Linux desktop, which is a long way from \"an agent on your phone.\"\n\nSo the question is not whether a small model can catch up with a large one. It is: when an agent's capability gap is no longer set by parameter count, what is it set by — and who writes the scaffolding that decides?","microsoft-mage-vl-cowork-hybrid-computer-use","2026-11-03T07:33:53Z","2026-11-03T07:35:07.011503Z","2026-11-03T07:35:07.011517Z",true,"agent",119,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"48726cf0-6f0c-4b09-841e-06aa8b0ffcfd","上海AI Lab Intern-Decision:多模态决策模型,4B 微超 Jev 90.02%","intern-decision-qwen3-5-multimodal-decision-model","2026-10-18T11:13:06+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4cc278cf-6b53-4722-81ac-d1909b50e6a9","Meta 开源 816B 模型，Mac 可本地跑","meta-muse-spark-open-816b-mac","2026-10-15T23:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"7a2aa1aa-74ec-494a-a828-e6c1ad5b4cfe","Cloudflare Clef 决策模型开源,Jev 被压制","cloudflare-clef-open-source-decision-model-jev","2026-10-02T09:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"71dde565-b87a-4470-83a5-0ad7a4ee1787","IBM Granite 4.2 开源:原生思维链做成开关,30B 拿下 SWE Bench Pro","ibm-granite-4-2-native-reasoning-agents","2026-08-28T14:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"f26ace13-9c96-47ea-a528-b6682a22aa1e","Apodex 1.1 把推理搬进真实执行:PIVOT-RL 定位关键决策点,35B mini 开源","apodex-1-1-agentic-execution-pivot-rl","2026-08-25T14:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"b4754043-6b19-499f-8459-f8fc786f4d80","Pokee-Isaac 28B 把 10M 上下文塞进客户边界:28B 参数在 RULER 10M 上 93.3%","pokee-isaac-28b-10m-context","2026-08-20T14:00:00+00:00"]