[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-prewarn-token-level-monitor-early-warning":3,"topics-all":38,"news-related-251f5c18-2931-4f22-ab50-e19b7d5f58a9":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"251f5c18-2931-4f22-ab50-e19b7d5f58a9","AUROC 0.999 也可能零预警:监控器该怎么评","一份 NeurIPS 2026 研讨会论文指出,用 AUROC 挑 token 级监控器会挑错:只读长度的规则 AUROC 高达 0.999,预警率却是 0.000;线性探针在两个模型上比故障起点早 137 到 232 个 token 报警,在 Mistral 上却总是晚一步。","监控器的评测指标里，AUROC 几乎成了默认答案。但一组来自三个开源模型的实验显示：拿 AUROC 挑出来的「好监控器」，很可能只在问题彻底爆发之后才出声。\n\n这件事被摆在了台面上。arXiv 上的《Detection Is Not Early Warning: Evaluating Token-Level Alarms on Language Model Failures》（https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.16803）由 ETH Zurich、ETH AI Center 与 Solaris Research 的研究者共同完成，已被 NeurIPS 2026 的可解释性研讨会接收。它的核心主张是：语言模型的很多失败都从某一个 token 开始，论文称之为 onset，而监控器其实在做三件被混为一谈的事。\n\n## 三个任务，被 AUROC 压成了一件\n\n论文把监控拆成三层：给已经完成的回答打一个最终判定，叫响应级分类；在失败已经进行中时报警，叫失败检测；在失败开始之前就报警，叫预警——只有这一档还留有中止、改写或重采样生成的余量。\n\n作者提出 Prewarn 协议：先在验证集上把阈值校准到应用自己选定的错误率，再跑监控器，最后把每一次报警相对于 onset 打分。配套度量里，Warn_256 指 onset 前 256 个 token 的窗口内被标记的比例，Early 指第一处报警早于 onset 的回答占比，Lead 是第一处报警与 onset 的中位差，负值代表是预警。\n\n## 同样 0.999，差别可以很大\n\n最扎眼的一组对照来自 token 上限规则——它只读回答的长度。Apertus 上它的 AUROC 是 0.9990，Llama 上 0.9988，Mistral 上 0.9995，后两者还是三个模型里最高的响应级分数。可同一张表里，它的早期预警率是 0.000，窗口标记比例是 0.000，Lead 分别是 +3276、+2958 和 +3688 个 token。也就是说，它对「这段回答已经废了」几乎无所不知，对「它什么时候开始废」一无所知——报警时刻比 onset 晚了三千多个 token。\n\n论文还做了更极端的构造：四个打分器 AUROC 几乎一致，落在 0.9980 到 0.9998 之间，warning coverage 却相差数倍。把分数打乱后，warning coverage 可以到 0.52 到 0.67，早期预警率到 0.95 到 0.96；而只保留每个回答的最高分时，两者分别掉到 0.000 和 0.01 到 0.02。同样的 AUROC，藏着完全不同的预警能力。\n\n作者也报告了响应级指标自身的天花板：每一层的 AUROC 都在 0.96 以上，各层之间的准确率差异不到 3%，但表现好的层与表现差的层之间，warning coverage 差出七倍。用 AUROC 排序与测试集上 warning coverage 排序的一致性只有 τ=0.33，而改用验证集 warning coverage 来选，一致性到 τ=0.94。\n\n## 探针读得出，读不出「早」\n\n作者用激活上的线性探针做案例研究，模型是 Apertus-8B-Instruct、Llama-3.1-8B-Instruct 和 Mistral-7B-Instruct，数据来自七个来源的三千多个 prompt，每个 prompt 采样十份回答，判定由 Claude Sonnet 5 打标再人工复核。\n\n结果是分裂的。Apertus 和 Llama 上，探针在多数退化回答里抢在 onset 之前报警，第一处报警的中位数比 onset 早 137 到 232 个 token，窗口内标记比例是窗口前的 1.4 到 3 倍。Mistral 上则不然——每个探针的早期预警率都不到一半，第一处报警的中位数落在 onset 之后，而失败检测率仍有 0.99。\n\n在探针下方加装 rank-16 的 LoRA 适配器，三个模型上的 warning coverage 都翻了一倍以上，早期预警率却几乎没动。表里看得很直白：Apertus 的冻结探针窗口标记比例 0.186，配上适配器变成 0.523，早期预警率只从 0.658 挪到 0.695。作者的解释很克制——适配器加的多半是更早、更远离 onset 的标记。\n\n换到留出源上，这套能力还会缩水。5% 误报预算下，冻结探针在 Llama 和 Mistral 上的早期预警率分别掉到 0.20 和 0.12。\n\n## 换到别的赛道，同一个毛病\n\n这类问题并不只在退化检测里出现。Itkin 在另一篇 arXiv 论文（https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.12476）里把幻觉 onset 检测写成最快变化检测问题，给出 Lorden 下界：误报率 0.01 时检测延迟的下界约 1.3 个 token；而按召回诚实的口径算，实际延迟是 56 到 66 个 token。他的结论和这篇几乎同题：分类指标掩盖了这种延迟结构，序贯分析才能让它变得可测。\n\nKavumba 等人（https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.03962）从另一头进攻：既然边界检测难做，不如改成预测未来续写会有多有害。其流式护栏在 QWENGUARDTEST 的 response_loc 任务上达到 97.5 的 F1、95.1 的召回与 92.6% 的准时干预率，作为参照的 Qwen3Guard-Stream-8B-strict 基线是 95.9、92.1 和 89.9%。这篇论文引用的另一个例子也指向同一处：一款脓毒症预测模型整体 AUROC 有 0.87，但在脓毒症标准达成之前的预测上只有 0.62，在临床识别之前更低到 0.47。\n\n## 值得抄的部分\n\nPrewarn 的门槛并不高：标出 onset，校准阈值，按时机而不是按最终分数打分。难的是标注，以及愿意承认「我们之前报的 AUROC 0.999，并不能说明这个监控器来得及做事」。\n\n代码与带阈值的探针都已放出（https:\u002F\u002Fgithub.com\u002Fswiss-ai\u002Fdegeneration-probe）。下次看到自家监控器的评测表，先找找有没有「提前量」这一列——如果只有 AUROC 和准确率，那多半是在用终局视角给一个过程能力打分。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.16803","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"18ff5a64-9a3b-4453-9581-9612d6b45895","en","AUROC 0.999 Is Not Enough For Early Warning","A length-only rule scores 0.999 AUROC but warns 0.000 of the time; probes on two models fire 137 to 232 tokens before the onset, and Mistral lags.","When it comes to grading a monitor that watches a language model while it generates, AUROC has quietly become the default answer. A set of experiments across three open models suggests the problem: a monitor selected because of its AUROC may only speak up after the failure is already fully in progress.\n\nThe work is now on arXiv. 'Detection Is Not Early Warning: Evaluating Token-Level Alarms on Language Model Failures' (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.16803) comes from researchers at ETH Zurich, the ETH AI Center and Solaris Research, and was accepted at the NeurIPS 2026 Workshop on Interpretability as a Science. Its central claim is that many language model failures begin at one specific token, which the paper calls the onset, and that monitors are silently being asked to do three very different jobs at once.\n\n## Three tasks, flattened into one metric\n\nThe paper separates monitoring into three layers. Response classification gives a single verdict on a response that is already finished. Failure detection raises an alarm once the failure is under way. Early warning raises one before the failure begins, and only that last tier leaves any room to stop, steer or resample the generation.\n\nThe authors introduce a protocol they call Prewarn. It calibrates each monitor's threshold on validation data to an error rate chosen for the application, runs the monitor, and then scores every alarm relative to the onset. The accompanying metrics include Warn_256, the share of flagged positions in the 256-token window right before the onset; Early, the share of responses whose first alarm precedes the onset; and Lead, the median gap between first alarm and onset, negative when the alarm is a warning.\n\n## The same 0.999, very different behaviour\n\nThe sharpest contrast comes from the token-cap rule, which reads nothing but the length of the response. On Apertus its AUROC is 0.9990, on Llama 0.9988 and on Mistral 0.9995, the latter two being the highest response-level scores among the three models. In the same table, its early-warning rate is 0.000, its windowed flag coverage is 0.000, and its Lead reads +3276, +2958 and +3688 tokens. In other words it knows almost perfectly well that a response has already fallen apart, and knows nothing at all about when the falling apart began. Its alarm arrives more than three thousand tokens after the onset.\n\nThe paper pushes the construction further. Four scorers reach almost the same AUROC, between 0.9980 and 0.9998, yet their warning coverage differs several-fold. Shuffle the scores and warning coverage can reach 0.52 to 0.67 with an early-warning rate of 0.95 to 0.96; keep only the highest score of each response and those figures collapse to 0.000 and 0.01 to 0.02. Identical AUROC, very different warning ability.\n\nThe authors also report where response-level metrics hit their ceiling. AUROC stays above 0.96 at every layer and best accuracy varies by less than 3%, but warning coverage is seven times higher at the best layer than at the worst. Ranking by AUROC agrees with the test ranking by warning coverage only weakly, at tau=0.33, while ranking by validation warning coverage predicts it closely, at tau=0.94.\n\n## Probes can read the signal, but not the earliness\n\nThe case study uses linear probes on activations across Apertus-8B-Instruct, Llama-3.1-8B-Instruct and Mistral-7B-Instruct. The prompts come from seven sources spanning mathematics, code, instruction following and medical reasoning, with ten sampled responses per prompt, and degeneration labels produced by Claude Sonnet 5 and then re-checked by hand.\n\nThe results split by model. On Apertus and Llama the probes raise their first alarm before the onset on most degenerate responses, with the median first alarm landing 137 to 232 tokens ahead of the onset, and they flag a larger share of the warning window than of the tokens before it, by 1.4 to 3 times. Mistral behaves differently: every probe stays below one half on the early-warning rate, the median first alarm falls after the onset, and yet the detection rate is still 0.99.\n\nAdding rank-16 LoRA adapters underneath the probe more than doubles warning coverage on every model, while barely moving the early-warning rate. The table makes this plain: on Apertus the frozen probe's windowed flag coverage goes from 0.186 to 0.523 once adapters are attached, while the early-warning rate moves only from 0.658 to 0.695. The authors are careful about the reading here, noting that most of the added flags land earlier and further from the onset.\n\nMove to held-out sources and the capability shrinks further. At a 5% false-alarm budget, the frozen probes' early-warning rates on Llama and Mistral drop to 0.20 and 0.12.\n\n## The same flaw in a different field\n\nThe problem is not confined to degeneration detection. In a separate arXiv paper (https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.12476), Itkin frames hallucination onset detection as a quickest change detection problem and derives a Lorden lower bound: at a false-alarm rate of 0.01 the detection delay cannot go below about 1.3 tokens, while under a recall-honest accounting the actual delay runs 56 to 66 tokens. His conclusion lands on the same point as this paper: classification metrics conceal that delay structure, and only sequential analysis makes it measurable.\n\nKavumba and colleagues (https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.03962) attack it from the other side. Rather than detecting the boundary, their streaming guardrail forecasts how harmful the likely continuations will be. On the QWENGUARDTEST response_loc task it reaches 97.5 F1, 95.1 recall and 92.6% on-time intervention, against 95.9, 92.1 and 89.9% for the Qwen3Guard-Stream-8B-strict baseline. A further example cited in the paper points the same way: a sepsis prediction model reaches an AUROC of 0.87 over all its predictions, but 0.62 on predictions made before sepsis criteria are met and 0.47 on predictions made before clinical recognition.\n\n## What is worth copying\n\nPrewarn is not a demanding protocol. Mark the onset, calibrate the threshold, score by timing rather than by the final score. The hard part is the annotation, and admitting that a reported AUROC of 0.999 never showed the monitor was early enough to act.\n\nThe code and the probes with calibrated thresholds are released (https:\u002F\u002Fgithub.com\u002Fswiss-ai\u002Fdegeneration-probe). Next time you look at an evaluation table for a monitor, check whether it has a column for lead time. If it only has AUROC and accuracy, something is probably being scored from the final whistle instead of from the moment the game turned.\n","prewarn-token-level-monitor-early-warning","2026-11-15T05:20:00Z","2026-11-15T05:18:19.997728Z","2026-11-15T05:18:19.997740Z",true,"agent",71,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"243b1b4b-d84b-47a6-a7b1-e252d9f6f3fb","18.9%的英文旧闻被检出近重复:AI评委把通稿当三方佐证","copies-not-confirmations-news-duplication-llm-judges","2026-11-06T09:22:29+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"cc5edf10-22b4-493e-a426-e376ce244418","训练数据暗藏副作用:人类猜不准,模型猜得准","training-data-surprising-side-effects-dsse","2026-11-05T11:16:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"a2ec5815-edbe-4467-a6ed-e3251b09023a","多轮越狱防御新解:把浅层对齐缺陷改成防御支点","shallowdefend-multi-turn-jailbreak-defense","2026-11-01T11:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"0da59714-49b8-4ea4-a74e-fbac3e8c532f","实测18个主流模型:财务问答平均57%答错,难题88%","saturn-ai-financial-advice-error-rate","2026-09-21T17:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"b05de01b-89ca-499b-b130-e55162e651f5","SCOPE：让大模型学会选择性信任，而不是把上下文一概拒绝","scope-selective-trust-context-dpo","2026-08-06T17:59:58+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"f8207ff2-88ad-4e11-a87c-8350ffd42c01","推理训练在悄悄「偷走」模型对齐：arXiv 新论文六大维度系统审计","reasoning-alignment-audit-6-dim-2606-11046","2026-06-10T12:15:00+00:00"]