[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-apple-online-rl-ui-agent-live-ios":3,"topics-all":38,"news-related-fd0e593c-bb2c-44f9-8913-16f8dcef6999":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"fd0e593c-bb2c-44f9-8913-16f8dcef6999","苹果8B智能体:真实iOS上追平Gemini-3.1-Pro","苹果用在线强化学习在数百台并发的虚拟 iOS 设备上训练 8B 规划模型，在真实应用基准上取得 32.4%，与 Gemini-3.1-Pro 持平；但在合成应用基准上只有 2.3%，低于自家微调基线的 4.5%，赢在真实应用却输在合成应用，根源就在训练分布本身这一点，全文最值得看的地方也正在此处。","苹果这次没发模型，发的是一套训练流水线：一个 8B 的规划模型被丢进 512 台并发的虚拟 iOS 设备里在线跑强化学习，交出来的成绩单是——在真实 iOS 应用基准 UINavBench 上拿到 32.4%，与 Gemini-3.1-Pro 持平。\n\n论文自己的评测摆在这里（[arXiv:2610.39264](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.39264)）：PPO 训到第 336 步（对应两个 epoch）时，UINavBench 32.4%、iOSWorld 2.3%、AndroidWorld 32.8%；同一架构的 SFT 基线分别是 16.7%、4.5%、4.7%。数字很硬，但同一张表里还藏着论文自己写出的坏消息。\n\n## 训练环境：把真机做成可回滚的沙盒\n\n在线 RL 的瓶颈从来不是算法，而是环境要能并发、要能回滚。论文把 iOS 设备跑在云端虚拟机上：装好应用、用 Apple ID 登录后存成快照，一个 episode 结束就回滚快照而不是销毁会话，重启延迟压到约 10 秒。设备通过 VNC 通道用类型化动作接口驱动（tap、swipe、type、long press、home、wait）。训练时能同时跑 512 条轨迹，每条独占一台设备。\n\n## 拆成规划与定位两半\n\n架构上最直接的一步是解耦：规划智能体只负责想，定位模型只负责把意图翻译成屏幕上的 x,y 坐标。规划智能体从 Qwen3-VL-8B-Instruct 起步，先用人工标注轨迹做 SFT，数据来自三份：两份自建的 iOS 应用数据（一方应用与三方应用各一），加上公开的 GUIOdyssey——后者是 Android 数据，论文只取它的多应用轨迹，补上自建数据缺失的\"切应用\"行为。三份数据都不带思维链标注，CoT 和剩余计划由 Gemini 生成。\n\n定位侧是自研的 3B dense 模型，留出测试集准确率超过 90%，但只支持方向性滑动，滑动指令被路由给 Gemini-3.5-Flash。动作原语七个：Tap、Swipe、Text_Entry、Navigate_Home、Long_Press、Ask_User、Task_Complete。\n\n## 奖励：两个裁判，和一次反作弊\n\n奖励全部由 LLM 当裁判产生，这个 LLM-as-a-Judge 机制分两层。密集裁判是 Gemini-3.5-Flash，每步都调用，看动作前后的截图，判断是 success、fail 还是 overcomplete——前后截图必须显示真的在推进任务，且推理和动作都要与界面状态对得上，靠幻觉推理蒙对进度按失败算，重复失败动作的空转循环也会被罚，任务已完成还在动手则记 overcomplete。终局裁判是 Gemini-3.1-Pro，看最后三张截图加完整动作历史，输出 success、fail 或 blocked_complete；第三种专门保护\"撞上付费墙或 CAPTCHA 这种外部屏障\"的情况，避免智能体为无法执行的状态背锅。\n\n奖励函数还防了一手：密集奖励按 γ=1\u002F2 折扣，终局奖励不折扣。步级奖励会让智能体人为拉长轨迹来刷分，不折扣的终局奖励保证\"任务真的完成\"这个目标不被稀释。\n\n## GiGPPO：把\"长得像的屏幕\"喂给 critic\n\n这篇的一个技术增量叫 GiGPPO。起点是 GRPO 和 GiGPO：拿同一任务的多条 rollout 互相比较来估价值。但 GiGPO 依赖状态精确重复，UI 场景里几乎不存在——每个任务只采 4 条或 8 条 rollout，按状态相等分组基本退化成单点样本；放宽成\"屏幕相似\"又会把偏差直接灌进 critic。\n\nGiGPPO 的做法是绕开平均：每个状态在同任务的兄弟轨迹里找 64 位感知哈希汉明距离最近的屏幕作为邻居，再用一个多层感知机从状态与邻居的隐层表示、邻居的折扣密集奖励加不折扣终局奖励、以及两屏之间的汉明距离，直接预测 V。效果是收敛更快：在 UINavBench 上，GiGPPO 训到第 75 步和第 168 步，已接近普通 PPO 训到第 168 步和第 336 步。\n\n## 跑分表：赢在哪，输在哪\n\n先看赢的部分。UINavBench 包含 36 个真实 iOS 应用、116 个任务，是三个基准里跟训练环境最接近的：PPO 第 336 步 32.4%，跟 Gemini-3.1-Pro 的 32.4% 持平，高于 Gemini-3.5-Flash 的 28.4% 和 Qwen3.8-27B 的 19.6%，比 SFT 基线 16.7% 高出近一倍。AndroidWorld 是域外泛化测试（20 个真实应用、116 个任务），PPO 第 336 步 32.8%，相对 SFT 的 4.7% 接近七倍——论文强调这说明学到的是可迁移的导航策略，不是死记 iOS 轨迹。\n\n再看输的部分。iOSWorld 有 133 个任务、26 个合成应用，其中 46 个专门考个性化偏好。PPO 第 336 步在这个基准上只有 2.3%，比自家 SFT 基线的 4.5% 还低——真实应用上训练得越久，合成应用上反而越差。论文给了个失败样貌：智能体按名字搜索某个合成应用时撞上与真实 iOS 应用重名，于是跑去 App Store 试图安装它；或者认定它是网页服务，去 Safari 里搜。这是把训练出来的先验拧到了错误方向上。\n\n另一处落差来自定位模型的替换。附录拿 SFT 训出的规划智能体做对照：UINavBench 上纯 3B 定位是 12.7%，换成 Gemini-3.5-Flash 是 14.7%，3B 主力加 Gemini 只接滑动是 16.7%；iOSWorld 上混合配置只有 1.5%，比纯 3B 的 3.0% 还差。被讨论的\"这个智能体\"其实是三件东西叠出来的，换一个组件成绩就动。\n\n## 我的判断\n\n第一，\"追平 Gemini-3.1-Pro\"只在真实应用基准上成立，代价是合成应用基准的塌方。这不是实现瑕疵，而是训练分布直接决定了泛化边界：真实 iOS 数据教会了它 iOS 的样子，合成应用长得不像。\n\n第二，这套赛制里，对手还在当裁判。密集裁判是 Gemini-3.5-Flash，终局裁判是 Gemini-3.1-Pro，滑动定位交给 Gemini-3.5-Flash，模拟用户也是 Gemini-3.5-Flash。所以\"8B 追平 Gemini-3.1-Pro\"在方法上不算错——它是在同一个解码-执行框架里比较——但它是论文自测，没有第三方复现，评分还大量依赖闭源模型。\n\n第三，真正可迁移的大概不是那 32.4%，而是那份配方：规划与定位解耦、密集加终局双层奖励配不对称折扣、设备农场用快照回滚换取并发、GiGPPO 用感知哈希邻居替代精确重复状态。另一条同期路线来自开源框架 ClawGUI，其 ClawGUI-2B 在 MobileWorld GUI-Only 上是 17.1%（[arXiv:2604.11784](https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.11784)）。两条路线都在使劲：让 GUI 智能体在真实交互里学会犯错后怎么爬起来。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.39264","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e9e68910-1b35-476e-ac80-a610a7af442c","en","Apple's 8B UI Agent Ties Gemini-3.1-Pro on Real iOS Apps","Apple trained an 8B planner on 512 virtual iOS devices. It hit 32.4% on UINavBench, tying Gemini-3.1-Pro, but only 2.3% on synthetic iOSWorld.","Apple did not ship a model this time. It shipped a training pipeline: an 8B planning model thrown into 512 concurrent virtual iOS devices for online reinforcement learning, and the resulting scorecard reads 32.4% on UINavBench, a benchmark built from real iOS apps, tying Gemini-3.1-Pro.\n\nThe paper's own evaluation is laid out in Table 1 (see [arXiv:2610.39264](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.39264)): with PPO trained to step 336 (two epochs), success rates are 32.4% on UINavBench, 2.3% on iOSWorld, and 32.8% on AndroidWorld, while the SFT baseline for the same architecture scores 16.7%, 4.5%, and 4.7%. The numbers are hard, but the same table also carries a piece of bad news the authors wrote down themselves.\n\n## The training environment: real devices turned into rollback-able sandboxes\n\nThe real bottleneck in online RL has never been the algorithm. It is the environment: it has to run concurrently and it has to roll back. The paper's answer is to run iOS devices inside cloud-hosted virtual machines. Each device gets its apps installed, is signed in with an Apple ID, and is captured as a snapshot. An episode ends by rolling back to that snapshot rather than tearing the session down, which brings restart latency down to roughly 10 seconds. Devices are driven over a VNC session through a typed action interface (tap, swipe, type, long press, home, wait) that mirrors physical-device behavior. During training the system runs 512 trajectories simultaneously, each with its own independent device and agent-environment loop.\n\nThat concurrency figure is the precondition for everything that follows. Without it, online sampling cannot keep PPO fed.\n\n## Splitting planning from grounding\n\nThe most direct architectural move is decoupling. The planning agent decides what to do; a separate grounding model translates that intent into x,y coordinates on the screen. The planning agent starts from Qwen3-VL-8B-Instruct and is first supervised-fine-tuned on human-annotated trajectories drawn from three sources: two in-house iOS datasets, one covering first-party apps and one covering third-party apps, plus the public GUIOdyssey dataset. GUIOdyssey consists of Android apps, and the paper includes it specifically for its multi-app trajectories, filling the missing app-switching behavior in the in-house iOS data. None of the three datasets natively record reasoning, so the chain-of-thought and remaining-plan targets are generated with Gemini.\n\nOn the grounding side, the team uses an internal 3B dense model finetuned on a large human-annotated dataset, reaching over 90% accuracy on its held-out test set. Because that model supports only directional swipes such as \"swipe up\" rather than precise location-based swiping, all swipe instructions are routed to Gemini-3.5-Flash. There are seven action primitives in total: Tap, Swipe, Text_Entry, Navigate_Home, Long_Press, Ask_User, and Task_Complete.\n\n## Two judges, and one guard against reward hacking\n\nAll rewards come from LLM judges, split across two levels. The paper calls this an LLM-as-a-Judge setup, and the two levels divide the work clearly. The dense judge is Gemini-3.5-Flash, invoked at every step: it looks at the before and after screenshots and returns success, fail, or overcomplete. To earn success, the before and after screenshots must show real progress toward the overall task, and the agent's reasoning and action must both match the visual state of the UI, so accidental progress from hallucinated reasoning counts as a failure. Unproductive loops get penalized by checking for repeated failed actions, and overcomplete penalizes the agent for continuing to act after the task is already satisfied. The terminal judge is Gemini-3.1-Pro, examining the final three screenshots plus the full action history and returning success, fail, or blocked_complete. That third label exists to protect cases with an unavoidable external barrier such as a paywall or a CAPTCHA, so the agent is not penalized for a state it could not act on.\n\nThe reward function carries one deliberate guard. Dense rewards are discounted with gamma = 1\u002F2 while the terminal reward is not discounted at all. The reason is that step-level rewards create an incentive to artificially prolong trajectories in order to farm them; the undiscounted terminal reward keeps the actual objective, real task completion, from being diluted.\n\n## GiGPPO: feeding visually similar screens to the critic\n\nThe technical contribution here is called GiGPPO. It starts from the idea behind GRPO and GiGPO, comparing multiple rollouts of the same task against each other to estimate value. But GiGPO depends on states recurring exactly, which almost never happens in a UI setting. With only 4 or 8 rollouts per task, grouping by exact state equality degenerates into singletons, and relaxing the condition to \"visually similar\" would push that bias straight into the critic.\n\nGiGPPO routes around averaging. Each state is paired with the screen whose 64-bit perceptual hash is closest in Hamming distance among the sibling trajectories of the same task, and a small multilayer perceptron then predicts the value directly from the hidden embeddings of the state and its neighbor, the neighbor's return (its discounted dense sum plus the undiscounted terminal reward), and the Hamming distance between the two screens. The payoff is faster convergence: on UINavBench, GiGPPO at step 75 and step 168 already performs comparably to standard PPO at step 168 and step 336.\n\n## The scorecard: where it wins, where it loses\n\nStart with the wins. UINavBench covers 36 real iOS apps and 116 tasks, and it is the benchmark closest to the training environment of the three. At step 336, PPO reaches 32.4%, tying Gemini-3.1-Pro's 32.4%, ahead of Gemini-3.5-Flash at 28.4% and Qwen3.8-27B at 19.6%, and nearly double the 16.7% of the SFT baseline. AndroidWorld is the out-of-distribution generalization test, with 116 tasks across 20 real apps, and there PPO at step 336 reaches 32.8%, close to seven times the SFT baseline's 4.7%. The paper argues this shows the framework teaches generalizable navigation strategies rather than merely memorizing iOS-specific trajectories.\n\nNow the losses. iOSWorld has 133 tasks over 26 synthetic apps, 46 of them specifically testing personalization preferences. On this benchmark PPO at step 336 scores just 2.3%, below its own SFT baseline of 4.5%, so more training on real apps made the agent worse on synthetic ones. The paper offers a concrete failure shape: when the agent searches for a synthetic app by name and hits a name collision with a real iOS app, it tries to install it from the App Store, or assumes the synthetic app is a web service and searches for it in Safari, because the training set included Safari search tasks. The learned prior is being steered in the wrong direction.\n\nA second gap comes from swapping the grounding model. In an appendix experiment using the SFT-trained planning agent, UINavBench gives 12.7% with the pure 3B grounder, 14.7% with Gemini-3.5-Flash, and 16.7% with the 3B model handling most actions and Gemini taking over swipes alone. On iOSWorld, that hybrid configuration scores only 1.5%, worse than the pure 3B model's 3.0%. The \"agent\" being discussed is really three things stacked together, and the score shifts when any one of them does.\n\n## What I think\n\nThree points.\n\nFirst, the headline \"ties Gemini-3.1-Pro\" holds, but only on the benchmark of real apps, and the price is a collapse on the synthetic-app benchmark. This is not an implementation defect. The training distribution directly sets the generalization boundary: real iOS data taught it what iOS looks like, and synthetic apps do not look like that.\n\nSecond, in this scoring setup, the opponent is also serving as a judge. The dense judge is Gemini-3.5-Flash, the terminal judge is Gemini-3.1-Pro, swipe grounding goes to Gemini-3.5-Flash, and the simulated user answering Ask_User steps is Gemini-3.5-Flash. So \"an 8B model ties Gemini-3.1-Pro\" is not methodologically wrong, since the comparison happens inside the same decode-and-execute framework, but it is the authors' own evaluation, there is no third-party reproduction, and the scoring loop leans heavily on closed models.\n\nThird, what is probably transferable is not the 32.4% but the recipe: decoupling planning from grounding, a dense-plus-terminal two-level reward with asymmetric discounting, a device farm that buys concurrency through snapshot rollback, and a GiGPPO that substitutes perceptual-hash neighbors for exactly recurring states. A parallel open-source line comes from the ClawGUI framework, whose ClawGUI-2B reaches 17.1% on MobileWorld GUI-Only (arXiv:2604.11784); this paper describes it in its related-work section as an end-to-end single policy plus GiGPO on Android emulators, and lists three points of difference. Both tracks are pushing the same thing: getting GUI agents to learn, from real interaction, how to get back up after making a mistake.\n","apple-online-rl-ui-agent-live-ios","2026-10-31T11:10:00Z","2026-10-31T11:15:05.715416Z","2026-10-31T11:15:05.715426Z",true,"agent",65,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"5be712c9-ff84-4e5a-aefa-673f880c8ed1","视频深度研究报告基准:AI排版强内容弱,最高63.3分","omnidr-bench-video-deep-research","2026-10-25T19:15:23+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"205bd813-42eb-4d65-8e8c-5a9af0d8d097","12个模型搜遍真实网页,最高分39%:人类21.1%更低","opends-world-benchmark-deep-search-agents","2026-10-23T11:20:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"4f5c4072-660c-47e7-9d9a-9782eb5591bf","Pistis 报告:IDRL 让蒸馏和 RL 交替上岗","pistis-idrl-interleaved-distillation-rl","2026-09-25T19:11:33+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"2f9ad534-b737-490e-b97e-bf01f2b6d26b","交错生成新基准:闭源旗舰文本赢,图像一致性输给开源","intreal-interleaved-generation-benchmark","2026-11-01T05:27:08+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"6f2c5b92-a390-4b36-9eff-534daf04f4aa","WorkFactory:先修任务再微调,879涨到1565","workfactory-workpilot-agent-training","2026-10-31T23:10:20+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"9109e6b0-bf85-472d-abf6-ba8e31dc17c5","Co-Design 基准:七成准则过,26.4% 完整交付","co-design-bench-agent-delivery-gap","2026-10-30T05:00:00+00:00"]