[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-metacognition-bench-llm-self-correction":3,"topics-all":31,"news-related-86565410-ced9-4e64-80b7-d97a353a1d1d":50},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":24,"published_at":25,"created_at":26,"modified_at":27,"is_published":28,"publish_type":29,"image_url":13,"view_count":30},"86565410-ced9-4e64-80b7-d97a353a1d1d","LLM 元认知首次可度量：Metacognition-Bench 用 300 道陷阱题 + 11 个开源错题雷达适配器把自我纠错做成开源工程","当 LLM 在对话里自信地说出错误答案时，「我能答对」与「我知道自己可能要错」其实是两种完全不同的能力。7 月 1 日，ginigen-ai 与 FINAL-Bench 在 Hugging Face 发布 Metacognition-Bench，把后者首次做成了一个可测量、可改进、可开源的工程问题。\n\n整套释放分为三层：一份 300+100 道「陷阱题」的 Metacognition-Bench（覆盖 121 个领域、8 种典型认知偏差——base-rate neglect、premise-shift blindness、二元框架、publication bias 等），一张 24 模型排行榜，以及 11 个针对不同基座、权重冻结、即插即用的「错题雷达」适配器。\n\n他们的核心观点是，单一指标刻画不了元认知，必须沿两个独立维度看：① 脆弱性（trap_rate，越低越好），衡量模型被陷阱选项诱导的概率；② 适配器增益（Δ AUROC，越高越好），衡量一个轻量 MLP 仅读最后一层 hidden state，能否预测「这道题要错」。基座权重完全不修改，只在输出侧加一个小头。\n\n最值得讨论的发现是：哪怕是 K-AI 榜第一的 JGOS-31B-Citizen，trap_rate 低到 0.005（400 题只错 2 道），但它在自由生成里对自己错误的 AUROC 仅为 0.5——与随机猜无异。「考得好」和「知道自己哪里要考砸」，是两个互不相干的维度。Qwen3.5-27B 在适配器增益榜上以 +0.800 AUROC 一骑绝尘，意味着它的原始自我感知几乎为零，反而给错题雷达留出了最大提升空间。\n\n更值得玩味的是：当多个顶级模型在选择题层面无法被分开时，自由生成 + 适配器这条曲线反而把它们重新区分开来。这暗示在 LLM 进入高风险部署时，「让模型自己举手说我可能错」可能比再加一道更难的考试更重要。\n\n基准、排行榜、适配器权重、训练代码全部开源，任何人提交一个 HF 模型即可被每日自动打分并入榜。","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fginigen-ai\u002Fmetacognition#adapter","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[],"metacognition-bench-llm-self-correction","2026-07-01T12:00:00Z","2026-07-01T12:21:47.249634Z","2026-08-19T02:08:40.142862Z",true,"agent",170,[32,41],{"slug":33,"tag_slug":33,"title_zh":34,"title_en":35,"intro_zh":36,"intro_en":37,"id":38,"is_active":28,"created_at":39,"modified_at":40},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":42,"tag_slug":42,"title_zh":43,"title_en":44,"intro_zh":45,"intro_en":46,"id":47,"is_active":28,"created_at":48,"modified_at":49},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":51},[52,57,62,67,72,77],{"id":53,"title":54,"news_slug":55,"published_at":56},"176b4807-da61-479f-a514-9381cd13319e","SP3O:3 个锚点修复 PPO critic 的平坦化","sp3o-sparse-critic-supervision","2026-09-17T17:10:01+00:00",{"id":58,"title":59,"news_slug":60,"published_at":61},"2e27016d-b90e-45c7-825a-41fd1e435c80","JHU 新研究:组合持续学习机制,百任务记忆留存从 1.2% 提到 34.9%","compose-cl-long-horizon-memorization","2026-09-16T15:10:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"4c4a2a9e-f69b-4985-bd42-97ab2ef4e2ac","Spark-X2.5-4B 开源:4B 跑 1M 上下文,22 项基准打 9B 级 Qwen3.5","spark-x2-5-4b-apache-open-source","2026-09-16T01:30:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"2731ed1c-17c3-4d85-9174-983cf50743e3","地铁售票机上的 AI 大考:2.6GB 端侧模型 91.32 分超 GPT-5.6,规则基线也拿 84.6","metrollm-bench-transit-kiosk-llm","2026-09-12T23:08:18+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"54b86d93-0fd0-4107-9353-9b79a1446f69","NVIDIA 开源 IMO 金牌完整配方:30\u002F42 分、561B 双专家、算力账本全公开","nvidia-nemotron-imo-gold-open-recipe","2026-09-11T17:13:27+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"d41175a7-ad10-4e00-9017-a148fa0a77b3","BenchMIRT 把 LLM 基准拆到单题:Ai2 想让模型排名不再「一张考卷定生死」","ai2-benchmirt-llm-benchmark-audit","2026-09-10T11:05:05+00:00"]