[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-web-agent-hallucination-mechanistic-interpretability-probe":3,"topics-all":38,"news-related-19dd65ff-646c-4edf-bd90-98b2dc8bf215":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"19dd65ff-646c-4edf-bd90-98b2dc8bf215","Web Agent幻觉可解释性:探针能预警,归因却失灵","一篇乔治梅森大学与特拉维夫大学团队的论文,用机制可解释性工具解剖Web Agent的幻觉失败:线性探针在动作写出前就以0.80的AUC预警,沿该方向干预可修掉85.4%的幻觉,但组件归因最好只修好8.9%,换个提示格式修正率就掉到58.1%。","真实网页上的 Agent 会犯一种很特别的错：它要点的按钮，页面上根本没有。\n\n作者来自乔治梅森大学（George Mason University）和特拉维夫大学（Tel Aviv University）。他们用 Online-Mind2Web 在真实网站上采集轨迹，动作空间限制在 CLICK、SCROLL、TYPE、HOVER 四种，模型是 Qwen2.5-7B-Instruct。最终 300 个 episode、2,071 个动作，32.6% 是幻觉。\n\n典型例子：任务是去 CarMax 找一辆 2022 年款特斯拉 Model 3，模型输出 TYPE 动作去填搜索框，但它的 selector 值在页面上找不到对应元素。日志透镜分析显示，这类幻觉来自模型「锁定了一个策略」：页面所有 Search、Explore、Find 并不存在，模型却生成了只引用虚构元素的动作。\n\n## 探针能在动作写出来之前报警\n\n论文在残差流的三个位置训练线性探针：prompt_end（模型还没吐出任何 token）、selector_start、selector_value_start。按 episode 切 70\u002F15\u002F15，得到 1,433 个训练动作、282 个开发、356 个测试。\n\nprompt_end 的探针在第 18 层拿到 0.80 的测试 AUC——一个线性方向在动作被写出来之前，就能分辨出「这条会不会翻车」。到 selector_value_start 时信号更强，从第 0 层起就很清楚。\n\n## 干预有效，解释无效\n\n沿探针方向做表征 steering，数字很漂亮：α=40 时修掉 85.4% 的幻觉，只误伤 7.0% 的正常动作，最有效的向量取自 selector_start 的 L17。\n\n但作者接着尝试「解释」，结论就变了。此前针对 RAG 幻觉的 ReDeEP 研究发现，残差流里幻觉常与 FFN 贡献压过注意力头输出有关。论文照此逐个放大各层 FFN 贡献（10 倍）、给注意力头注入高斯噪声，再做联合干预。\n\n最好的组合是放大第 22 层全部注意力头、同时把第 22 到 24 层的 FFN 贡献减半——修好 8.9%，误伤 2.4%。加上第 27 层注意力头，修好率掉到 5.9%，误伤却飙到 27.6%。更麻烦的是被单独干预出来的幻觉跟自然幻觉不一样：放大第 15 层 FFN 会把选择器值改写成 VAR-1000000000 这种语法无效的乱码。\n\n激活修补那边，动作列表确实是因果贡献者：在第 27 层恢复干净状态，能把被污染的概率质量救回 0.36，而选择器起点和任务描述两栏都不到 0.02。但知道它有影响，并不告诉你哪一部分在驱动模型改主意。作者据此判断：计算摊在多个组件和位置上，单点归因接不住。\n\n## 换个格式就掉链子\n\n最能说明问题的一组对照在附录里。同一个 steering 向量，只把候选动作列表从 JSON 数组换成带编号的 markdown 列表，效果立刻变差：\n\n- α=10：修好率 43.9% 掉到 19.4%\n- α=20：75.6% 掉到 41.9%\n- α=40：85.4% 掉到 58.1%，少了约 27 个百分点\n\n误伤率几乎没动，α=40 时两边都是 7.0%。作者的解释是：这个方向纠缠的是动作格式，而不是「幻觉方向」本身。\n\n## MI 的三个假设在这里失效\n\n标准 MI 方法依赖的三个假设，在智能体场景里都不成立：\n\n- **局部性**：假设小范围组件能解释绝大部分行为，而这里是分布式计算，单点干预修不动。\n- **平稳性**：激活修补需要一条干净基线，但智能体每走一步页面就变、上下文就长。论文构造配对反事实，删掉选中的合法动作逼模型出幻觉，只有 38% 的情况真能造出来，且一步内轨迹就发散。\n- **时间局部性**：决策本该落在某个时间步上，实际上在生成之前就已经在残差流里成型并增强。\n\n预训练 SAE 也没帮上忙。Qwen2.5-7B-Instruct 上的 BatchTopK SAE，字典 131,072 个特征、每 token 激活 k=64，用在第 27 层——因果恢复最高的那层——排最前面的特征所激活的 token，跟幻觉行为基本无关。论文把「面向智能体的 MI 基准还是空白」列为路线图第一条。\n\n## 两条独立的旁证\n\n一是 arXiv:2509.18970，一份 LLM 智能体幻觉的独立综述：它按智能体工作流的不同阶段给幻觉分了类，梳理了 18 类触发原因。另一支团队从分类学一侧给出的侧写，和主论文指向同一件事——Agent 幻觉是独立于普通 LLM 幻觉的一类问题。\n\n二是 arXiv:2603.28744，Barin-Pacela 等人发表在 UAI 2026（PMLR 337:364-412）的工作 Stop Probing, Start Coding。他们用受控实验拆解 SAE 在组合式分布漂移下的失败，结论是瓶颈在字典学习而非推理过程。两支团队方法不同，却指向同一条规律：探针和 SAE 找出来的方向会跟着输入表面形式跑偏。\n\n## 我的看法\n\n这篇论文的价值在于它是一个少见的诚实结果。而最容易被误读的是：0.80 的 AUC 和 85.4% 的修好率看起来像「可解释性已经能修 Agent 了」，但那个修好率对提示格式敏感——改一下动作列表的写法，护栏就少掉近三成能力。而真实系统里改提示格式成本几乎为零：加几行 few-shot、把 JSON 换成表格，都可能顺手改掉它。这套探针更稳的用法是当观测器报警，而不是当纠正器改写输出。\n\n再说方法论。0.80 AUC 说明模型内部存在一个「准备这么干」的可分方向，但可分不等于机制。这篇论文最锋利之处在于它没把可分当成找到机制，反而把「换个格式就失效」摆出来，提醒你很多被当成机制的线性方向，可能只学会了表面格式。\n\n所以路线图上那几条——在智能体轨迹上重训 SAE、跨时间步追踪模型何时锁定策略、建面向智能体的 MI 基准——不是宏大收尾，而是失败清单逼出来的补丁。\n\nAgent 的失败很少是某一行代码写错了，更常见的样子是：几十步里慢慢决定了要干一件网页上根本不存在的事。可解释性如果只在出错之后解释得通，那它对还没出错的那一步还是黑箱。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18514","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ba3f6ef4-5326-45bf-bae9-84fb3760d3b6","en","Web Agent Hallucination: Probe Warns, Attribution Fails","George Mason and Tel Aviv dissect web agent hallucination: a probe warns at 0.80 AUC, steering fixes 85.4%, but attribution manages 8.9%.","Real web agents fail in a very specific way: they click buttons that do not exist on the page.\n\nThe authors, from George Mason University and Tel Aviv University, collected interaction trajectories on live websites using Online-Mind2Web. They restricted the action space to four verbs, CLICK, SCROLL, TYPE and HOVER, following the original Mind2Web setup, generated a list of candidate actions from each page, and asked the model to pick from that list. The model under test was Qwen2.5-7B-Instruct. The final dataset consists of 300 episodes and 2,071 actions, of which 32.6% are hallucinated, meaning the action the model generated does not appear in the list at all.\n\nA typical example: the task is to find a 2022 Tesla Model 3 on CarMax. The model emits a TYPE action meant to fill in a search box, but its selector value does not correspond to any element on the current page. Logit lens analysis shows that these hallucinations mostly come from the model locking onto a strategy. Actions such as Search, Explore or Find do not exist on the page, yet the model generated syntactically well-formed actions referencing these fictional elements.\n\n## A probe can raise the alarm before the action is written\n\nThe paper trains linear probes independently at three positions of the residual stream: prompt_end, the point before the model emits a single token; selector_start; and selector_value_start. The split is episode-level at 70\u002F15\u002F15, yielding 1,433 training actions, 282 development actions and 356 test actions.\n\nThe probe at prompt_end reaches a test AUC of 0.80 at layer 18. Put differently: a linear direction can already tell apart cases that will go wrong before the action is written. The signal only strengthens further along. At selector_value_start it is already clear from layer 0 and keeps improving.\n\n## Intervention works, explanation does not\n\nRepresentation steering along the probe direction produces excellent numbers. At alpha=40 it fixes 85.4% of hallucinations while disrupting 7.0% of valid actions, and the most effective vector comes from L17 at the selector_start position.\n\nBut when the authors turn to explanation, the conclusion flips. Prior work on RAG hallucination, the ReDeEP line of research, found that hallucinations in the residual stream are often attributable to FFN contributions overshadowing attention head outputs. The paper therefore amplifies each layer's FFN contribution by 10x one at a time, injects Gaussian noise into individual attention heads, and then runs joint interventions.\n\nThe best configuration amplifies all attention heads at layer 22 while halving FFN contributions at layers 22 through 24, fixing 8.9% while corrupting 2.4%. Adding layer 27 attention heads drops the fix rate to 5.9% while the break rate jumps to 27.6%. More awkwardly, the hallucinations induced by isolated intervention look nothing like natural ones. Amplifying the FFN at layer 15 rewrites a legitimate selector value into syntactically invalid garbage such as VAR-1000000000, rather than the orderly hallucination of referencing an element the page does not have.\n\nThe activation patching result is more nuanced. The action list is indeed a causal contributor: restoring a clean state at layer 27 recovers 0.36 of the corrupted probability mass, while the corresponding figures for selector start and the task description both stay below 0.02. But knowing that the action list matters does not tell you which part of it drives the model to change its mind. The authors conclude that the computation here is spread across multiple components and positions, and single-point attribution cannot handle it.\n\n## Change the format and the whole thing falls apart\n\nThe most telling comparison sits in the appendix. Keeping the same steering vector and changing only the prompt format, so the candidate action list goes from a JSON array to a numbered markdown list, degrades the result immediately:\n\n- At alpha=10: the fix rate falls from 43.9% to 19.4%\n- At alpha=20: 75.6% falls to 41.9%\n- At alpha=40: 85.4% falls to 58.1%, roughly 27 percentage points\n\nThe break rate barely moves, sitting at 7.0% on both sides at alpha=40. The authors read this as the direction being entangled with action format rather than with the hallucination direction itself.\n\n## Three assumptions of MI break down here\n\nThe paper attributes this to the fact that the assumptions standard MI methods rely on do not hold in agentic settings.\n\nThe first is localizability, the assumption that a small set of components explains most of the behavior. Here the computation is distributed, and single-component interventions barely move it. The second is stationarity. Activation patching needs a clean baseline, but an agent changes the page and grows its context at every step, so no two steps share a comparable baseline. The paper tries to construct paired counterfactuals by removing the selected valid action to force a hallucination, and only 38% of cases actually produce one, with the trajectory diverging within a single step. The third is temporal locality. A decision is supposed to live at one timestep, but here it is already forming in the residual stream before generation begins and strengthens progressively.\n\nPretrained SAEs do not help either. The BatchTopK SAE for Qwen2.5-7B-Instruct has a dictionary of 131,072 features with a sparsity of k=64 active features per token, and it was applied at layer 27, the layer with the highest causal recovery in the activation patching results. The features that most strongly separate hallucinated from valid cases promote tokens largely unrelated to the observed hallucination behavior. The paper lists the absence of MI benchmarks for agent applications as the first item on its roadmap, one reason being that nobody agrees on what counts as a faithful, complete and actionable explanation.\n\n## Two independent corroborations\n\nThe first is arXiv:2509.18970, an independent survey of hallucinations in LLM-based agents. It taxonomises agent hallucinations by the different stages of the agent workflow and examines eighteen triggering causes. A different team working from the taxonomy side arrives at the same picture as the main paper's observations: agent hallucination is a category of problem distinct from ordinary LLM hallucination, and it deserves a benchmark of its own (https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.18970).\n\nThe second is arXiv:2603.28744, work by Barin-Pacela and colleagues published at UAI 2026 (PMLR 337:364-412), titled Stop Probing, Start Coding. Through controlled experiments they decompose SAE failures under compositional distribution shift and conclude that the bottleneck is dictionary learning rather than the inference procedure. The two teams use entirely different methods but point at the same regularity: directions found by linear probes and SAEs drift along with the surface form of the input (https:\u002F\u002Farxiv.org\u002Fabs\u002F2603.28744).\n\n## Where I come out\n\nMy read is that the value of this paper lies in how honest the result is, and in how directly the conclusion constrains engineering practice.\n\nStart with what is easiest to misread. An AUC of 0.80 and an 85.4% fix rate look like interpretability has already learned to repair agents. But that fix rate is sensitive to prompt format: change how the action list is written and the guardrail loses close to thirty percent of its capability. And in a real system, changing the prompt format costs almost nothing. Adding a few few-shot lines, reordering the candidate list, swapping JSON for a table, any of these can shift it by accident. So the more reliable use for this probe is as an observer raising alarms, not as an automatic corrector rewriting outputs.\n\nOn methodology: an AUC of 0.80 does show that a linearly separable direction meaning the model is preparing to act this way exists internally, but separability is not mechanism. What is sharp about this paper is that it refuses to treat separability as mechanism discovery, and instead puts the fact on the table: the same direction loses its effect once the format changes, which is a warning that many directions treated as mechanisms may just have learned surface format.\n\nThat is why the roadmap items, retraining SAEs on agent trajectories, tracking across timesteps when the model commits to a strategy, and building MI benchmarks for agents, are not the kind of grand finale researchers like to write. They are patches squeezed out of this list of failures.\n\nAgent failures are rarely one line of code written wrong. Far more often they look like this: over dozens of steps, the agent slowly decides to do something that does not exist on the page. If interpretability only explains things after they go wrong, then for the step that has not gone wrong yet, it is still a black box.\n","web-agent-hallucination-mechanistic-interpretability-probe","2026-11-17T03:15:00Z","2026-11-17T03:22:49.544878Z","2026-11-17T03:22:49.544898Z",true,"agent",1,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"9649aae9-2aef-4668-96e8-d8ced4378a71","专家头衔一贴，模型就多改口：Claude掉16.1点","llm-expert-disagreement-revision-selectivity","2026-11-15T07:10:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"9785c9bb-85a0-4b97-8890-aa2d6808bf86","Agent记忆陷阱:良性对话把不安全操作率推到81%","agent-memory-trap-benign-overgeneralization","2026-11-08T15:12:52+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9e209ccb-cd2c-44cd-acaf-b168454074b2","中国 AI 智能体也会撒谎:88% 轮次出现虚假陈述","china-ai-agents-lie-88pct-tender-sim","2026-10-04T04:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4977d1f4-c8f7-480c-aaa1-ec01d69f44d8","8B 拿 SFT+GRPO 打 685B MoE:KaliBench 把 LLM 网络安全工具调用拆到命令行级","kalibench-cybersecurity-cli-runtime-rewards","2026-10-03T05:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9ebb888c-dfe7-416a-9940-a913527d4f73","AI Agent 的失败比成功更值钱:5 万对错误诊断数据,修正通过率 18.4%→51.1%","agent-error-dataset","2026-10-01T15:11:08+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"3bcb0e1d-99ea-4fae-9bdd-b6b625aabf10","代码 agent 8 成都在骗你:12 模型实测揭晓","overclaimbench-llm-agents","2026-09-21T07:00:00+00:00"]