[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-focusgui-omni-modal-gui-agents":3,"topics-all":38,"news-related-ace7a537-e2de-448f-811f-a028357aed42":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"ace7a537-e2de-448f-811f-a028357aed42","FocusGUI:给全模态GUI智能体装上注意力滤镜","GUI 智能体被截图、音频、视频等全模态输入包围,无关信号累积成分心误差,把轨迹带偏。南洋理工与阿里巴巴的 FocusGUI 只训一个轻量适配器:按当前决策动态挑相关证据,压缩后注入冻结解码器,主干不动。4 个全模态底座上 Type Match 平均提升 33.2,Exact Match 升至 3 倍。","GUI 智能体正在从\"看截图点按钮\"进化成\"开着摄像头、听着麦克风、读着屏幕\"的全模态操作者。但输入通道越多,麻烦也越多:决定下一步点击的可能只是当前这一帧截图,而上一段音频早就过期了。南洋理工大学与阿里巴巴团队 10 月 19 日提交至 arXiv 的 FocusGUI 给出了一个轻量解法,并冲上 Hugging Face 论文日榜第一,拿到 181 个赞。\n\n## 问题:分心是会复利的\n\n论文把痛点命名为 Sequential Distraction(序列性分心):每一步都把全部模态证据均匀喂给模型,无关信号就成了下一步的噪音;更糟的是,执行过的动作会回灌进上下文,错误逐步复利,最终把整条轨迹带偏到不可逆。作者由此提出 Focused Perception 三原则——步级自适应、动作充分、非侵入。\n\n## 解法:只训一个适配器\n\nFocusGUI 的做法是给冻结的底座外挂一个轻量 adapter,流程分四步:通道分解(从当前观测拆出互补证据)、步级自适应优先级(对照当前决策参考给各模态通道加权)、紧凑聚焦(把加权证据压进低维隐向量)、残差对齐(在动作生成边界注入零初始化的残差)。主干、Transformer 解码器和原 LM head 全部冻结,动作以结构化 JSON 自回归生成。训练只用动作标注,既不需要模态相关性标签,也不用微调底座。\n\n## 效果:四个底座平均涨 33.2\n\n在 Qwen2.5-Omni-7B、MiniCPM-o-4.5、VITA-1.5、Baichuan-Omni-1.5 四个全模态底座上,论文报告平均 Type Match 提升 +33.2、Exact Match 达到冻结底座的 3.0 倍;对比现有最强增强方法,Type Match 也高出 +30.1。训练与评测基于 OmniGUI 数据集,官方代码已在 GitHub 开源,四个底座的适配器训练脚本全部放出。\n\n## 所以呢\n\n这波 agent 热潮的默认直觉是\"上下文越多越好\",FocusGUI 反着来:相关性是瞬态的,步级筛选比全量堆料更重要。对工程团队的直接启示是——没有预算全参微调全模态大模型时,冻结主干 + 外挂适配器 + 纯动作监督,是一条成本可控的路径。GUI 智能体距离可靠的日常助手,差的也许不是更大的模型,而是更干净的注意力。\n\n论文: https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.26203","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.26203","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"1175369e-b158-4caa-bc5c-7789f904af5a","en","FocusGUI: A Light Adapter That Keeps GUI Agents From Drifting","FocusGUI freezes omni-modal backbones, trains a light adapter on actions alone, and lifts Type Match +33.2 across four GUI-agent backbones.","GUI agents are evolving from screenshot-clicking tools into omni-modal operators that watch camera streams, listen to audio, and read the screen at once. More input channels, however, bring a new problem: the signal that decides the next tap might be the current frame only, while the previous audio clip has long expired. FocusGUI, submitted to arXiv on Oct 19 by a team from Nanyang Technological University and Alibaba Group, offers a lightweight fix — and it climbed to #1 paper of the day on Hugging Face with 181 upvotes.\n\n## The Problem: Distraction Compounds\n\nThe paper names the pain point Sequential Distraction: feeding all modal evidence uniformly at every step turns irrelevant signals into noise for the next action. Worse, executed actions feed back into the context, so errors compound step after step until the trajectory drifts irreversibly. The authors formulate the requirement as Focused Perception — step-adaptive, action-sufficient, and non-intrusive.\n\n## The Fix: Train Only an Adapter\n\nFocusGUI attaches a lightweight adapter to a frozen backbone in four stages: channel decomposition (split the current observation into complementary evidence), step-adaptive prioritization (weight each modality channel against a step-specific decision reference), compact focus (project the weighted evidence into a low-dimensional latent), and residual alignment (inject a zero-initialized residual at the action-generation boundary). The backbone, Transformer decoder, and original LM head all stay frozen, and actions are generated autoregressively as structured JSON. Training relies on action annotations alone — no modality-relevance labels, no backbone fine-tuning.\n\n## Results: +33.2 Type Match on Average\n\nAcross four omni-modal backbones — Qwen2.5-Omni-7B, MiniCPM-o-4.5, VITA-1.5, and Baichuan-Omni-1.5 — the paper reports an average Type Match gain of +33.2 and Exact Match at 3.0 times the frozen backbone, plus +30.1 Type Match over the strongest existing enhancement method. Training and evaluation use the OmniGUI dataset, and the official code with adapter scripts for all four backbones is open-sourced on GitHub.\n\n## So What\n\nThe default instinct in the agent boom is \"more context is better.\" FocusGUI argues the opposite: relevance is transient, and step-level selection beats full-context stuffing. For engineering teams, the takeaway is direct — when you cannot afford full fine-tuning of an omni-modal model, freezing the backbone and bolting on an action-supervised adapter is a budget-friendly path. What stands between GUI agents and reliable daily assistants may not be a bigger model, but cleaner attention.\n\nPaper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.26203","focusgui-omni-modal-gui-agents","2026-10-21T19:05:00Z","2026-10-21T19:06:34.751354Z","2026-10-21T19:06:34.751364Z",true,"agent",62,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"089f56f3-32ff-4036-89b5-728d5f5a9359","边聊边干活:腾讯混元开源全模态交互 Agent Gander,小脑管对话、大脑管执行","hunyuan-gander-omni-interaction-agent","2026-09-09T21:07:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"f60a52ae-76c9-4fdf-a1d7-efc41bd50c38","阶跃星辰发布Step 3.7 Flash：面向生产级Agent的开源基座模型升级","stepfun-step-3-7-flash-agent-multimodal","2026-05-29T01:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"601ab1a1-6704-4bd9-bd4e-81fe87366a25","DeltaStreamer开源:流式视频理解学会\"回头补证据\"","deltastreamer-streaming-video-replay","2026-10-21T17:11:14+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"aa718f3c-14cf-4984-8762-794eaaee8c16","MAI-Gym:9B 智能体在 2 万个并行环境里做在线 RL","mai-gym-gui-agent-online-rl-9b","2026-10-21T09:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a491d4db-2d5a-4723-b093-28d1512fd647","VLA-Δ:物理状态变化当监督信号,LIBERO 98.4","vla-delta-physical-state-transition","2026-10-20T19:07:27+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"b44fb160-9027-44cd-b355-a66869004abc","Nemotron 3.5 Super VL 开源:120B 总参只激活 12B","nemotron-3-5-super-vl-open-120b-moe","2026-10-20T17:11:56+00:00"]