[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-unified-training-single-likelihood-foundation-models":3,"topics-all":38,"news-related-41889a7a-a82d-4e8e-b693-7db85a4f62c7":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"41889a7a-a82d-4e8e-b693-7db85a4f62c7","CMU统一训练:数学分打平,下一轮代码反超分阶段","卡内基梅隆论文把预训练和强化学习合并成一个损失函数,分阶段训练这一做法本身并非必需,同批数据混进预训练末段后留出损失只由2.537升到2.540,而分阶段做法把留出损失推到12.150,数学分数接近打平但下一轮代码强化学习全面反超,作者同时承认实验规模小且未建立可预测的扩展律。","预训练、中期训练、监督微调、强化学习——今天几乎所有基础模型的训练流水线都长这样:先灌语料,再补领域数据,再对齐,最后上强化学习。卡内基梅隆大学与约翰霍普金斯大学的一篇新论文说,这套分段的算法切分可能并不是必须的:把强化学习改写成对模型自己生成分布的最大似然拟合之后,几个阶段可以共用同一个加权似然损失、同一个训练器。\n\n论文没有停在公式上。它在两个规模上做了完整对照:一个 124M 参数模型(20B token 预训练语料、510M token 中期训练语料),一个 0.76B 参数模型(240M token、24 个来源的混合语料,数学强化学习用 Math12K,过滤后剩 11977 条提示)。结论比\"统一之后更强\"要微妙得多——真正的差别不在当前这一轮的分数,而在留出的账和下一轮的起点。\n\n## 分阶段训练欠下的遗忘账\n\n先看最扎眼的一组数。在强化学习开始之前,预训练留出损失(ClimbMix 与 Dolmino 按 75\u002F25 加权)在完全不做中期训练时是 2.537,通用基准均值 42.37。分阶段做法把中期训练放在强化学习之前,峰值学习率取 2×10⁻⁴,跑 1 个 epoch,留出损失飙到 12.150,基准均值掉到 35.00;跑到 4 个 epoch 是 15.104 和 35.61。把峰值学习率压到 5×10⁻⁵、同样跑 1 个 epoch,留出损失也仍有 4.135,基准均值 38.49。\n\n也就是说,在论文自测的保留性对照里,分阶段中期训练把预训练语料的拟合水平打掉了好几倍,基准分数同时被拖低。而统一训练把同一批中期训练数据混进预训练的最后 8147 步(总 38147 步、每步 256 条 2048 token 序列、共 20000014336 token)之后,留出损失稳在 2.537 到 2.540 之间,基准均值 41.85 到 42.42,几乎贴着\"什么都不做\"的基线。\n\n## 真正的分水岭在下一轮训练\n\n如果只看数学任务,两种做法几乎打平。12000 步数学强化学习之后,中期训练混合比例从 0% 加到 5%,MATH pass@1 在 16.11 到 16.91 之间小幅波动,GSM pass@1 从 32.68 降到 30.80;同一张表里基座模型是 6.54 和 17.84,分阶段基线是 16.98 和 31.35,分阶段反而略高一点。留出损失这一侧倒是单调:0% 混合是 +0.0737(比基座更差),0.1% 就变成 −0.0154,5% 是 −0.1474,而分阶段基线是 +0.0472。也就是说,哪怕只混千分之一,遗忘都比两条基线都轻。\n\n代码能力的账要等到下一轮才结清。数学强化学习结束时,论文自己的对照里分阶段模型在 HumanEval+ 和 MBPP+ 上是 15.51 和 25.74,混了 5% 的统一模型是 10.90 和 20.97——看起来分阶段赢了将近五个点。接着给这批模型接上同一套代码强化学习:20000 道 KodCode-V1 的 Python 题,4000 步,评测覆盖 164 道 HumanEval+ 和 378 道 MBPP+。四个统一模型在两个基准的 pass@1 上全部反超,5% 那一档是 24.96 对 22.94、32.69 对 29.81。\n\n论文随后按总算力重做对比,估算覆盖预训练、中期训练与 RL 三段的模型 FLOPs:预算越大,统一训练在准确率和覆盖度上的优势越明显,在训练分布之外的题上优势最稳定。不过论文自己也提醒,这条前沿只刻画了测过的配置,既不是全局最优,也不构成有预测力的扩展律。\n\n## 三处需要泼冷水\n\n第一,这些数字全部来自论文自己的实验,目前没有第三方复现。论文划的边界也很清楚:算力对照并没有建立有预测力的 scaling law,优化超参如何随数据调度变化没有深入研究,多数实验一次只优化一个强化学习任务,论文还明确承认自己\"并未消除阶段这个概念\"——下游适配新任务终究还是要单独一轮训练。\n\n第二,那个吞吐差要打折看。生产日志记录统一训练 0% 混合时每步约 3.7 秒、分阶段强化学习每步约 28.8 秒,但两套系统分别用 24 张和 16 张 GPU,日志不含启动与重启开销,而且异步训练器可以复用已有响应,所以它量的是优化吞吐、不是生成新 rollout 的速率。\n\n第三,靠\"往基座拉\"能不能替代混数据?论文做了 KL 正则的对照:加大惩罚确实能压住留出损失的恶化,但最强的惩罚会明显砍掉数学准确率与覆盖度。论文的结论是,在他们测的范围内,没有一种惩罚能同时复现统一训练在数学表现、语料损失、SciQ 和 MMLU 上拿到的组合。\n\n工程入口是开放的:仓库 yudasong\u002Funified-training 放了训练器、生成器、实验脚本和第三方 NeMo-RL 集成,README 给出了三套环境的安装路径,要求 NVIDIA 驱动 580 及以上。论文的计算资源来自 ASCR Leadership Computing Challenge 项目 ERCAP0034861 与橡树岭、阿尔贡国家实验室设施。\n\n## 所以呢\n\n判断一份训练配方好不好,最容易被跑分带偏:这个配方在你今天测的那件事上分数最高。但这篇论文更值得记住的读数,是 2.537 与 12.150 之间那道裂缝,以及 HumanEval+ 上 15.51 变成 10.90、又在 4000 步之后变成 24.96 的那条曲线。前者是当下看不见的遗忘成本,后者是当下看错的起点。论文自己的说法放在这里挺合适——一个数据源的价值,取决于它在什么时候被引入,以及它怎么和后续学习相互作用。统一训练没有取消阶段,它只是把\"分阶段\"从一道架构决定,变成了一个你可以随时改的调度参数。\n\n论文全文:[Training Foundation Models with a Single Algorithm](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40564) · [项目页](https:\u002F\u002Fyudasong.github.io\u002Funified-training\u002F) · [代码仓库](https:\u002F\u002Fgithub.com\u002Fyudasong\u002Funified-training)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40564","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"bf507335-f93f-42ed-9aab-ffbb80ba1238","en","CMU Unified Training: Math Ties, Code Wins Next Round","CMU merges pretraining, midtraining and RL into one loss. Loss 2.537 vs 12.150 staged, unified wins later code RL.","Most foundation-model training pipelines today look roughly the same: pour in web text, top up with domain data, align, then run reinforcement learning. A new paper from Carnegie Mellon University and Johns Hopkins University argues that this staged separation is not actually fundamental. Rewrite reinforcement learning as maximum-likelihood fitting to the model's own generation distribution, and the stages collapse into one weighted likelihood loss handled by a single trainer.\n\nThe paper does not stop at formulas. It runs a full comparison at two scales: a 124M-parameter model (20B-token pretraining corpus, 510M-token midtraining corpus) and a 0.76B-parameter model (240M tokens from 24 components, with Math12K supplying the math RL tasks and 11977 prompts surviving length filtering). The conclusion is subtler than \"unified is better.\" The real difference shows up not in the current round's score, but in the bill that is deferred and in the starting point you hand to the next round of training.\n\n## The forgetting bill staged training leaves open\n\nStart with the most alarming set of numbers. Before RL even begins, held-out pretraining loss (ClimbMix and Dolmino combined at a 75\u002F25 weighting) is 2.537 when no midtraining happens at all, with a mean general-benchmark score of 42.37. Staged training puts midtraining before RL, at a peak learning rate of 2x10^-4, for one epoch: held-out loss jumps to 12.150 and the benchmark mean drops to 35.00. Four epochs gives 15.104 and 35.61. Lowering the peak learning rate to 5x10^-5 for one epoch still leaves held-out loss at 4.135 and the mean at 38.49.\n\nIn the paper's own retention comparison, then, staged midtraining wrecks the model's fit to the pretraining corpus several times over, and drags the benchmark score down with it. Unified training instead mixes the same midtraining data into the last 8147 pretraining updates (38147 updates in total, 256 packed sequences of 2048 tokens each, 20000014336 tokens processed). After that, held-out loss stays between 2.537 and 2.540 and the benchmark mean between 41.85 and 42.42, essentially level with the do-nothing baseline.\n\n## The real dividing line shows up in the next training round\n\nIf you look only at the math task, the two approaches are close to level. After 12000 steps of math RL, sweeping the midtraining mix from 0% to 5% moves MATH pass@1 within a narrow band of 16.11 to 16.91 while GSM pass@1 slides from 32.68 down to 30.80. In the same table, the base model is at 6.54 and 17.84, and the staged baseline is at 16.98 and 31.35, so staged actually edges slightly ahead. Retention, though, moves monotonically: 0% mixing gives +0.0737 (worse than the base model), 0.1% already gives -0.0154, and 5% gives -0.1474, against +0.0472 for the staged baseline. Even one part in a thousand of mixing leaves less forgetting than either baseline.\n\nThe coding bill only comes due in the next round. When math RL ends, the paper's own comparison puts the staged model at 15.51 on HumanEval+ and 25.74 on MBPP+, while the unified model mixed at 5% is at 10.90 and 20.97, so staged looks like it is ahead by nearly five points. Then every model gets the same code RL on top: 20000 Python problems from KodCode-V1, 4000 updates, evaluated over 164 HumanEval+ and 378 MBPP+ problems. All four unified models overtake staged on pass@1 for both benchmarks, and the 5% mixture posts 24.96 against 22.94 on HumanEval+ and 32.69 against 29.81 on MBPP+.\n\nThe paper then repeats the comparison under a total-compute accounting that estimates model FLOPs across pretraining, midtraining and RL. The larger the budget, the more clearly unified training leads on accuracy and coverage, and its edge is most stable on problems outside the training distribution. The authors caution, though, that the resulting frontier only characterizes the configurations they tested: it is neither a global optimum nor a scaling law with predictive power.\n\n## Three caveats worth stating\n\nFirst, every number here comes from the authors' own experiments, and there is no third-party replication yet. The paper draws its own boundaries plainly: the compute study does not establish scaling laws with predictive power, how optimization should change with the data schedule and mixture design is not studied in depth, most runs optimize a single RL task at a time, and the authors explicitly acknowledge that they do not fully eliminate the notion of stages, since adapting a foundation model to a new downstream task will inevitably require a separate training stage.\n\nSecond, discount the throughput gap. Production logs record roughly 3.7 seconds per update for unified training at 0% mixing versus 28.8 seconds per update for staged RL, but the two systems use 24 and 16 GPUs respectively, the logs exclude launch and restart downtime, and the asynchronous trainer can reuse existing responses, so what is measured is optimization throughput rather than the rate of producing fresh rollouts.\n\nThird, can regularizing toward the base model substitute for mixing data? The paper runs a KL-regularization comparison: raising the penalty does hold back the deterioration in held-out loss, but the strongest penalty visibly cuts mathematical accuracy and coverage. The conclusion is that within the range they tested, no penalty reproduces the combination of math performance, corpus loss, SciQ and MMLU accuracy that unified training achieves.\n\nThe engineering entry point is open: the yudasong\u002Funified-training repository ships the trainer, the generator, experiment scripts, and a third-party NeMo-RL integration, with the README documenting three separate environment setups and requiring an NVIDIA driver of 580 or newer. The compute came from ASCR Leadership Computing Challenge project ERCAP0034861 plus the Oak Ridge and Argonne national laboratory facilities.\n\n## So what\n\nJudging a training recipe is most easily misled by the scoreboard, since the recipe you are evaluating is the one that scores highest on the thing you happen to be testing today. The readings worth remembering from this paper are the gap between 2.537 and 12.150, and the curve where 15.51 on HumanEval+ becomes 10.90, then becomes 24.96 after 4000 more updates. One is a forgetting cost you cannot see in the moment; the other is a starting point you misread in the moment. The paper's own formulation fits neatly here: the value of a data source depends on when it is introduced and on how it interacts with subsequent learning. Unified training does not abolish stages. It turns \"staged\" from an architectural decision into a scheduling parameter you can change at will.\n\nFull paper: [Training Foundation Models with a Single Algorithm](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40564) - [project page](https:\u002F\u002Fyudasong.github.io\u002Funified-training\u002F) - [code repository](https:\u002F\u002Fgithub.com\u002Fyudasong\u002Funified-training)","unified-training-single-likelihood-foundation-models","2026-10-31T09:16:57Z","2026-10-31T09:17:12.277673Z","2026-10-31T09:17:12.277686Z",true,"agent",171,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"97df8600-9c52-46e9-9650-90c6d2b198f1","ARLoop:循环模型的残差流改成注意力","arloop-attention-residuals-looped-transformer","2026-11-05T03:19:37+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"ba729d8d-d18e-4e77-824e-1db99ec469eb","近零词嵌入暗坑:五成门控检查点训练静默白跑","near-zero-embeddings-gated-training-stall","2026-11-04T15:20:36+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"022731b7-2aa0-435f-b93e-3f84eea2699a","R2Orch双层自博弈:技能内化,零技能上阵","r2orch-self-play-skill-withdrawal","2026-11-02T21:10:07+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"eb251970-12c7-4edb-8de5-ef33e60f09fa","BYOD：3.2 GPU 小时把自回归模型改成扩散模型","byod-ar-to-diffusion-conversion-lora-single-gpu","2026-11-02T09:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"be62f4f6-6935-4c2c-abe8-9e0732bb04ef","Trajectory2Gym:智能体的成功轨迹不等于好教材","trajectory2gym-compositional-interaction-synthesis","2026-10-27T21:10:28+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d3807737-6e24-41cc-9012-9d7aeb5eed22","RL 训完不如基座:FORK 用精确率改写评分标准奖励","fork-rubric-rl-precision-reward","2026-10-27T05:10:19+00:00"]