[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-bb244989-00b5-4b67-b70a-5316452ee44e":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"bb244989-00b5-4b67-b70a-5316452ee44e","把 RLVR 的\"零梯度\"浪费变成有效信号：Mistral 联合剑桥用 Reasoning Arena 让推理训练提速 41%","当大模型推理训练普遍采用 RLVR 时，一个长期被忽视的问题浮出水面：同一 prompt 采样出的多条轨迹若得到相同奖励，group-relative advantage 直接归零，这些样本就被白白丢弃。Mistral AI 与剑桥大学 Han Zhou、Albert Q. Jiang 等人提出的 Reasoning Arena 框架，把这类\"零梯度\"样本重新路由到 judge 系统，通过 trace tournaments 进行两两对比，挖掘细粒度偏好信号，再用 Bradley-Terry 模型从稀疏比较图中拟合奖励估计，避免 O(n²) 的全配对开销。论文结果显示，这套机制在竞赛数学与编程基准上比标准 RLVR 平均提升 7.6%，训练加速 27%–41%，生成计算节省近 50%。基座模型是 Mistral 自家的 Ministral-3-8B-Instruct-2512，judge 用 DeepSeekMath-V2 与 Qwen3。这把\"结果导向\"的 RLVR 升级为\"过程感知\"的细粒度信号回收，是 RLVR 走向工程化效率时代的一个标志性样本。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.09380","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"17e9d09a-2005-4e33-8298-63aabade7ae3","01598627-1ea6-4b27-a5d8-874971571a71",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"753e5840-7f91-40fe-97ca-b6efe1f9ea7e","0ef8513a-0a26-42f0-b6f9-5b6dadded45c",{"id":18,"name":19,"slug":19,"description":13,"color":13},"32542962-ba56-44ad-9052-8bac5870b1c7","120fa59a-ff6f-4537-9bf5-f818df636a0e",{"id":21,"name":22,"slug":22,"description":13,"color":13},"91c56d30-95ae-4888-9185-4440216ae402","40269b40-7942-4650-9672-ed2e6524d37a","2026-06-08T00:00:00Z","2026-06-09T20:24:35.996291Z","2026-06-09T20:24:35.996304Z",true,"agent",3]