[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ai-peer-reviewer-error-injection-audit":3,"topics-all":39,"news-related-349ef1bf-0024-48b0-8b6e-61fd21993f86":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":25,"news_slug":32,"published_at":33,"created_at":34,"modified_at":35,"is_published":36,"publish_type":37,"image_url":14,"view_count":38},"349ef1bf-0024-48b0-8b6e-61fd21993f86","往真论文埋357个错:AI审稿人年检报告","LLM 审稿已进入生产规模,但谁来审审稿人?被 NeurIPS 2026 接收的一篇论文向 30 篇真实 arXiv 论文注入 357 个分级错误,实测 12 个 AI 审稿系统:同一句提示下三个前沿模型检出率相差约 50 个百分点,强制检索引用的多智能体系统 Skepthical 在天体物理卷拿到 70.6%。","AI 审稿已经不是概念验证,而是生产规模的现实——但审稿人自己靠不靠谱,一直缺一份系统体检。剑桥大学与 Flatiron 研究所等机构的五位研究者做了件较真的事:把 30 篇真实 arXiv 论文当考卷,人为埋进 357 个分级错误,让 12 个 AI 审稿系统来抓错。这篇论文已被 NeurIPS 2026 接收(https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17052),以下数字均为论文实测口径。\n\n## 考试怎么设计\n\n两套语料各 15 篇:天体物理卷取自 2026 年 4 月 22 日的 astro-ph 提交,机器学习卷取自 7 月 15 日的 cs.LG 分层抽样,两个日期都晚于所有参评模型的知识截止,杜绝「背过考卷」。错误分四类——通用问题、数学、数值、引用——各设三档严重度,由 agent 直接修改论文 LaTeX 源码,通过机械校验器后才算注入成功。裁决交给 Anthropic、OpenAI、Google 三家模型组成的跨家族评审组做多数表决,干净副本则单独测假阳性。\n\n## 换模型比换架构管用\n\n同一句「请以期刊审稿人口吻写审稿报告」,GPT-5.5、Claude Opus 4.7、Gemini 3.1 Pro 在天体物理卷的总检出率分别是 60.5%、36.7%、11.9%,差出 48.6 个百分点;机器学习卷为 63.9%、43.9%、11.1%。把多智能体审稿系统 Skepthical 的全套操作指令压成 978 行结构化提示后,弱模型被大幅拉起:Gemini 提升 25.4 到 40.0 个点,Claude 提升 16.6 到 28.3 个点。头部配置对轻微档注入也能检出 57% 到 73%,弱配置连公然错误也大量漏掉。\n\n## 工具只有强制执行才会被用\n\nClaude Code 与 Skepthical 同样握有 arXiv 访问权,但执行轨迹显示 Claude Code 从未真的去取引用文献;Skepthical 把检索设为必经步骤后,天体物理卷引用错误检出率 46.7%,其余系统至多 26.7%。同一模型、同一沙箱下,仅把一句话提示换成结构化指令,Claude Code 总检出率在天体物理卷提升 32.2 个点、机器学习卷 38.9 个点——结构化配置写了 21 个 Python 校验脚本,自由配置一个都没写。\n\n## 两个意外\n\n其一,在审稿数据上微调的两个开源权重审稿器,在主场机器学习卷分别只检出 1.7% 与 2.8%,几乎交白卷。其二,数学类举报格外不可靠:前沿系统汇总后,天体物理卷 30%、机器学习卷 67% 的数学标记被裁决为误报,约占总假阳性的一半。假阳性整体可控:除两个开源权重审稿器外,所有系统在天体物理卷都低于 9%;Skepthical 两卷分别为 2.8% 与 4.0%。\n\n## 数字之外\n\n成绩上限也没多高:Skepthical 在天体物理卷检出 70.6%,仍漏掉约三成;机器学习卷上单模型结构化 GPT-5.5 反而以 81.1% 领先,差异不显著。论文也自我打折:Skepthical 用了三家供应商的模型,并非等成本对照,优势不能全归功于管线设计。成本侧,托管系统单篇 0.34 到 2.41 美元:Skepthical 单篇 2.41 美元、约 24 分钟,单模型结构化配置 0.55 美元、约 2 分钟。评测管线、注入错误目录与全部裁决日志已在 GitHub 开源(https:\u002F\u002Fgithub.com\u002Finigozubeldia\u002Fskepthical-benchmark)。\n\n## 所以呢\n\n急着上线 AI 审稿的会议与期刊,这份基准给了直接可抄的作业:先跑埋错体检再上线,选对模型比调架构重要,假阳性要单独报告——数学标记尤其不可信。对所有 agent 落地场景,结论同样成立:把验证做成可选项,agent 就永远不选;做成不可绕过的步骤,它才开始真的「读」文献。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17052","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,19,22],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":18,"color":14},"9112951a-2abb-4214-b63a-385ec7afb2ba","ai-for-science","AI for Science 专题：追踪 AI 在生命科学、化学材料、物理世界模型等科学方向的关键突破",{"id":20,"name":21,"slug":21,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":23,"name":24,"slug":24,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[26],{"id":27,"lang":28,"title":29,"summary":30,"content":31},"38f9cd6c-6357-4651-868e-7c298840fc89","en","357 Injected Errors: Auditing AI Reviewers on Real Papers","A NeurIPS 2026 study planted 357 errors in 30 arXiv papers to audit 12 AI reviewers: frontier models differ by 50 points; Skepthical tops at 70.6%.","AI peer review is no longer a demo; it runs at production scale. But who audits the auditors? Five researchers from the University of Cambridge, the Flatiron Institute and collaborating institutions turned 30 real arXiv papers into an exam: they planted 357 graded errors into the papers and let 12 AI reviewer systems hunt for them. All numbers below are the paper's own measurements. The work has been accepted at NeurIPS 2026 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17052).\n\n## How the exam is built\n\nTwo corpora of 15 papers each: an astrophysics set drawn from astro-ph submissions of 2026-04-22, and a machine-learning set sampled from cs.LG submissions of 2026-07-15. Both days postdate the knowledge cut-off of every evaluated model, so nobody can answer from memory. Errors come in four types — general issues, mathematics, numerics, citations — at three severity levels, injected by agents directly into the LaTeX source and accepted only after a mechanical validator passes. Verdicts come from a cross-family panel of Anthropic, OpenAI and Google models by majority vote, while clean copies are used to measure false positives.\n\n## Choosing the model matters more than the architecture\n\nGiven the same one-sentence prompt, GPT-5.5, Claude Opus 4.7 and Gemini 3.1 Pro reach total detection rates of 60.5%, 36.7% and 11.9% on astrophysics — a 48.6-point spread; on ML the numbers are 63.9%, 43.9% and 11.1%. Compressing the multi-agent system Skepthical's full operating instructions into a 978-line structured prompt lifts the weaker models: Gemini gains 25.4 to 40.0 points, Claude gains 16.6 to 28.3. Strong configurations still catch 57% to 73% of even the subtle injections, while weak ones miss most errors even when blatant.\n\n## Tools get used only when they are enforced\n\nClaude Code and Skepthical both hold arXiv access, yet the execution traces show Claude Code never actually retrieved a cited paper. Skepthical makes retrieval a mandatory step and reaches a 46.7% citation-error detection rate on astrophysics, against at most 26.7% for any other system. With model and sandbox fixed, swapping the one-sentence prompt for the structured instructions alone raises Claude Code's total detection by 32.2 points on astrophysics and 38.9 on ML — the structured configuration wrote 21 Python verification scripts, the free-form one wrote none.\n\n## Two surprises\n\nFirst, the two open-weights reviewers fine-tuned on review data detect almost nothing on their home ML corpus — 1.7% and 2.8%. Second, mathematics flags are especially unreliable: pooled over frontier systems, 30% of math flags on astrophysics and 67% on ML are judged false, and math flags account for about half of all false positives. Otherwise false positives stay contained: apart from the two open-weights reviewers, every system is below 9% on astrophysics, with Skepthical at 2.8% and 4.0% across the two corpora.\n\n## Beyond the scores\n\nThe ceiling is modest: Skepthical detects 70.6% on astrophysics, still missing roughly a third; on ML, structured single-model GPT-5.5 actually leads at 81.1%, a difference the paper calls not significant. The authors also self-discount: Skepthical draws on three providers and is not a capacity-matched comparison, so its edge cannot be credited to the pipeline design alone. On cost, hosted systems span 0.34 to 2.41 dollars per paper: Skepthical at 2.41 and about 24 minutes per paper, the structured single-LLM configuration at 0.55 and about 2 minutes. The evaluation pipeline, the injection catalogues and the full adjudication logs are open-sourced (https:\u002F\u002Fgithub.com\u002Finigozubeldia\u002Fskepthical-benchmark).\n\n## So what\n\nFor venues rushing to deploy AI reviewers, this benchmark offers a ready-made template: run an error-injection audit before launch, treat model choice as the main lever, and report false positives separately — math flags above all. The lesson generalises to every agent deployment: make verification optional and the agent will skip it; make it unavoidable and it finally starts reading the literature.\n","ai-peer-reviewer-error-injection-audit","2026-11-15T23:30:00Z","2026-11-15T23:09:35.149924Z","2026-11-15T23:09:35.149934Z",true,"agent",119,[40,48],{"slug":17,"tag_slug":17,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":36,"created_at":46,"modified_at":47},"AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":36,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"30df67dc-7c74-4a8c-80f7-12006096e7bc","AI做实验:94%到20%的置信度断层","sciresearch-eval-agent-scientific-judgment-interval-coverage","2026-11-18T01:40:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"158f2181-ecf3-4e5c-ac5a-16e5c45c9d9a","千篇论文建成AI实验室:前沿模型74%胜过人类作者","scientist-world-1000-research-environments","2026-11-08T21:08:04+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"87d43d7b-b32e-499d-af8a-68afc6506a2c","科研Agent大考:头名80.9分,达标率仅26%","sci-agent-exam-scientific-agent-benchmark","2026-10-26T17:09:05+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ec2c558c-502d-43a5-9494-c766dfd515e9","EurekAgent：把科学发现的瓶颈从「工作流」拽到「环境」，11 美元跑出 26 圆 packing 新 SOTA","eurekagent-environment-engineering-11-usd","2026-06-11T17:56:35+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"03403319-8597-4876-8661-fad74ee0ac48","屏幕上的警告,智能体最高92%照做","computer-use-agents-cross-screen-harm","2026-11-18T23:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"b9ca804d-7d7a-4241-a4a5-2882679279d4","同一模型同一温度,换个采样器解题率62%变0","long-context-generation-sampler-temperature-collapse","2026-11-18T03:30:00+00:00"]