[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-fork-rubric-rl-precision-reward":3,"topics-all":38,"news-related-d3807737-6e24-41cc-9012-9d7aeb5eed22":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"d3807737-6e24-41cc-9012-9d7aeb5eed22","RL 训完不如基座:FORK 用精确率改写评分标准奖励","CMU 与 KAIST 团队发现:用评分标准当强化学习奖励,模型会越训越啰嗦,精确率从 46.6 掉到 25.5,评测胜率只剩 3.1%,还不如没训过的基座。论文提出的 FORK 奖励把召回率与精确率合成为 Fβ,在 RubricHub-Medical 上把胜率从 10.6% 拉到 68.0%。","用评分标准(rubric)给模型打分很常见:每题配一组「该提到的要点」,命中越多、权重越高,奖励越高。问题在于这类奖励天然是召回率(recall)指标——只数「该说的说了没有」,不数「多说的该不该说」。一旦它被直接用作强化学习的奖励函数,这个结构性漏洞会被优化器放大到肉眼可见。\n\n## 一轮 RL 之后的反直觉结果\n\n论文在 RubricHub-Medical 测试集上给出很扎心的数字。基座是 Qwen2.5-7B-Instruct,基线是论文采用的标准覆盖率奖励。训练后宏平均召回率从 22.6 涨到 35.7,覆盖更全;但宏平均精确率从 46.6 掉到 25.5,平均回答长度从 2,432 涨到 9,195。胜负率更刺眼:训好的模型与没训过的基座正面对比,赢面只有 3.1%。一轮 RL 跑完,在评审模型眼里反而是原始模型更受欢迎。\n\n这也不是某个基线独有的问题。论文复现的四个覆盖率类奖励,胜率都落在 3.1% 到 10.6% 之间,没一个接近五成。\n\n## 病根:九成的评分标准只会说「要提」\n\n论文引用的统计显示,RubricHub 里 90.2% 的评分条目是「必须提到某点」式的存在型条目,只有 8.6% 是「不许提到某点」式的缺失型条目。存在型条目只有加分方向,没有减分方向。于是覆盖率奖励训练时同时干两件事:把该说的补上(存在型满意度 +15.0 个百分点),也把不该说的塞进去(缺失型满意度反而 -3.2 个百分点)。模型学会「多写」,因为在这样的奖励函数里多写永远赚。\n\n## 精确率是怎么定义的\n\n论文把回答切成句子级单元,对每个单元算两件事。一是必要性:每条评分条目有总额度为 1 的信用预算,分给所有回应它的单元;若 K 个单元提到同一条,每个只拿 1\u002FK 信用,冗余被自动打折,累积信用越过阈值的单元才算「必要」。二是事实正确性:独立地给每个单元打对错标签,只有它表达的事实主张成立才算对。\n\n单元只有既必要、又事实正确才计入精确率,精确率是这类单元占全部单元的比例。最后用 Fβ 把精确率与召回率合成单一奖励:β>1 偏向召回,β\u003C1 偏向精确率。关键不在那个 β,而在信用按 1\u002FK 分配——它让「同一件事说三遍」拿不到分。\n\n## 换回 68% 胜率,代价写在召回率上\n\n换用这个奖励后,医疗域胜率从 10.6% 提升到 68.0%,在 HealthBench 这类外部基准上从 22.1% 提升到 74.1%,回答长度压到 3,533;论文另报告 β=1.25 一档相对标准覆盖率奖励把长度缩到约 1\u002F2.1。代价同样清楚:该方法召回率是 32.5,低于四个覆盖率基线 35.7 到 38.7 的区间。它没做到「既全又准」,只是把权衡曲线往另一侧推了一步。\n\n另外几条边界也得摆明。偏好型奖励基线 Holistic Preference 拿到 65.4% 胜率,接近 68.0%,只是召回率掉到 27.5——两条路是同一权衡的两端。换成另一套评审提示词做鲁棒性检验时,它的 89.4% 反倒高于 Fork 的 88.6% 和 84.6%。官方仓库只放出标准奖励、Generic Criteria 和 Fork 三种实现,论文其他基线与消融未一并释放,README 的论文链接仍是占位符。评审模型用 GPT-5.4,训练侧评分模型是 Qwen3.6-35B-A3B(与三个前沿模型判断的相关系数 0.714),策略模型只有 7B,场景限定在医疗与科学问答。原始论文见 [arXiv:2610.33512](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.33512)。\n\n## 奖励函数的形状,决定了模型最终学到什么\n\n这篇论文的价值不在于又涨了几个点,而在于它把「非可验证任务怎么做 RL」这件事的度量层拆开了:打分器算出来是召回率,优化器就照着召回率优化,多写永远赚,模型于是学会用废话和错误内容换分数。\n\n对做医疗、科研、法律咨询类助手的团队,这是直接的工程警告:指标看着合理、分数还在涨,但评审模型给出的偏好已经倒向原始模型。加一个精确率项,或干脆给「多写」标上成本,可能比换更大的评审模型更管用。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.33512","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"c407b2f2-d8ad-4c20-a192-a6d4db265930","en","Rubric RL Made Models Worse Than Base: FORK Adds Precision","Rubric-based RL rewards make models verbose and weaker than their base. FORK adds a precision term, lifting win rate from 10.6% to 68.0%.","Scoring a model with rubrics is becoming standard practice: every question carries a set of points that should be mentioned, and the more points a response hits (weighted by importance), the higher the reward. The problem is that such a reward is inherently a recall metric. It only counts whether required content is present, never whether the extra content should be there at all. When that reward is used directly as a reinforcement learning objective, the structural gap gets amplified by the optimizer until it is impossible to miss.\n\n## The counter-intuitive result after one round of RL\n\nThe paper reports a set of sobering numbers on the RubricHub-Medical test set. The base policy is Qwen2.5-7B-Instruct, and the baseline is the standard rubric-coverage reward the paper adopts. After training, macro rubric recall climbs from 22.6 to 35.7, so coverage improves. But macro precision falls from 46.6 to 25.5, and average response length grows from 2,432 to 9,195. The win rate is the sharpest number of all: measured head to head against the untrained base model, the RL-trained model wins only 3.1% of comparisons. After a full round of RL, the judge model actually prefers the original model.\n\nThis is not peculiar to one baseline. Across the four rubric-coverage rewards the paper reproduces, win rates span 3.1% to 10.6%, with nothing close to fifty percent.\n\n## Why the root cause sits in the rubrics themselves\n\nA statistic cited in the paper shows that 90.2% of the RubricHub scoring entries are presence-based items of the form \"must mention X\", while only 8.6% are absence-based items of the form \"must not mention X\". Presence-based entries have a rewarding direction but no punishing direction. Coverage rewards therefore do two things at once during training: they fill in what should have been said (presence-based satisfaction rises 15.0 percentage points) and they stuff in what should not have been said (absence-based satisfaction actually drops 3.2 points). The model learns to write more, because in a reward function shaped this way, writing more always pays.\n\n## How the precision term is actually defined\n\nThe paper splits a response into sentence-level units and scores two properties for every unit. The first is necessity: each rubric entry gets a total credit budget of one, distributed among all the units that address it, so if K units touch the same entry each receives only 1\u002FK of the credit and redundancy is discounted automatically; a unit counts as necessary once its accumulated credit crosses a threshold. The second is factual correctness: every unit is independently labeled correct or not, and it counts as correct only when the factual claims it expresses actually hold.\n\nA unit contributes to precision only if it is both necessary and factually correct, and precision is the fraction of such units among all units. Finally, precision and recall are combined into a single reward through their weighted harmonic mean, the F-beta score: beta above 1 leans toward recall, beta below 1 leans toward precision. The load-bearing detail is not the beta but the 1\u002FK credit split, which is what stops a model from saying the same thing three times and collecting points for it.\n\n## Back to a 68% win rate, with the cost written on recall\n\nWith the new reward, medical-domain win rate rises from 10.6% to 68.0%, and on external benchmarks such as HealthBench it rises from 22.1% to 74.1%, while response length drops to 3,533. The paper separately reports that the beta=1.25 setting shortens length to roughly 1\u002F2.1 of the standard coverage reward. The cost is equally visible. Recall under Fork is 32.5, below the 35.7 to 38.7 range of the four coverage baselines. It does not achieve both completeness and precision; it shifts the trade-off curve in a different direction.\n\nSeveral boundaries are worth stating plainly. The preference-based baseline Holistic Preference reaches a 65.4% win rate, close to the 68.0% of Fork, but with recall dropping to 27.5, so the two approaches are two ends of the same trade-off. Under a different judge prompt for robustness testing, Holistic Preference posts 89.4% on HealthBench, edging out Fork at 88.6% and 84.6%. The official repository currently ships only three implementations, the standard reward, Generic Criteria, and Fork, so the remaining baselines and ablations in the paper are not released; the README still carries a placeholder link to the paper. Evaluation uses GPT-5.4 as the reference grader, the training-side grader is Qwen3.6-35B-A3B with a correlation of 0.714 against the judgments of three frontier models, the policy model is 7B, and the setting is limited to medical and scientific question answering. The paper is at [arXiv:2610.33512](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.33512).\n\n## The shape of the reward decides what the model learns\n\nThe value of this paper is not the extra points on a leaderboard. It takes apart the measurement layer of how to run RL on non-verifiable tasks: when the grader computes a recall, the optimizer optimizes a recall, writing more always pays, and the model learns to trade filler and factual errors for score.\n\nFor teams building medical, scientific, or legal-assistance models on top of large language models, this is a direct engineering warning. The evaluation metric looks reasonable and the score keeps going up, but the preference the judge model reports has already tilted back toward the base model. Adding a precision term, or simply putting an explicit cost on writing more, may do more good than swapping in a larger judge model.","fork-rubric-rl-precision-reward","2026-10-27T05:10:19Z","2026-10-27T05:23:55.098306Z","2026-10-27T05:23:55.098320Z",true,"agent",165,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"206ea36a-1eea-463f-a241-1e3b32f5ec2d","USTC GraphForge:证据图把任务和 rubric 钉在一起,Qwen3.6-27B 涨三基准","graphforge-ustc-qwen36-27b-evidence-graph","2026-10-04T03:05:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"8def771a-d936-4859-930d-02c3011dc55c","LimiX-2 开源：一个模型吃下分类回归插补，表格三榜登顶","limix-2-tabular-foundation-model","2026-09-17T21:09:27+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"176b4807-da61-479f-a514-9381cd13319e","SP3O:3 个锚点修复 PPO critic 的平坦化","sp3o-sparse-critic-supervision","2026-09-17T17:10:01+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"d41175a7-ad10-4e00-9017-a148fa0a77b3","BenchMIRT 把 LLM 基准拆到单题:Ai2 想让模型排名不再「一张考卷定生死」","ai2-benchmirt-llm-benchmark-audit","2026-09-10T11:05:05+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"5bfdf32b-44eb-4eb5-a98b-39e921168182","九天内连发五款前沿模型:7 月的大模型军备赛,真正决胜负的不再是 benchmark","july-2026-five-frontier-models","2026-07-23T12:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"8173a86b-4e5e-429a-8ddf-f98af527b4b5","LLM-as-a-Verifier：验证成 LLM 第四 scaling 维度","llm-as-a-verifier-fourth-scaling","2026-07-07T12:00:00+00:00"]