[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-trl-v1-13-tensor-core-lmhead-ppotrainer":3,"topics-all":35,"news-related-dcbb1042-4fef-4a6a-bd7a-0e11b4d97c10":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"dcbb1042-4fef-4a6a-bd7a-0e11b4d97c10","TRL v1.13.0:一处 dtype 修复拿回 21.6% kernel 时间,PPOTrainer 退役","后训练库 TRL 发布 v1.13.0:默认 chunked 损失的 lm_head 投影因多余 fp32 升精度占掉 21.6% kernel 时间,修复后 Qwen3-8B 微调吞吐 1.69 倍;新增百万 token 长上下文训练指南;2020 年首个 commit 落地的 PPOTrainer 被整体移除。","后训练库 TRL 发了 v1.13.0,更新日志里最值得注意的不是新功能,而是一次几乎让人脸红的修复:默认 chunked 损失的 lm_head 投影,此前一直先把两个操作数升到 fp32 再做矩阵乘,把这个最重的 GEMM 从张量核挤到了 SIMT 通路。官方在 8×H100 上对 Qwen3.6-35B-A3B 跑 `trl sft` 剖析,这两个 fp32 SIMT GEMM 占了全部 GPU kernel 时间的 21.6%。\n\n## 一处 dtype 决策,吃回两成 kernel 时间\n\n机理不复杂:`h.float() @ w.float().t()` 里两个操作数本来就是 bf16,升精度没有带来任何信息增益,反而每次都要物化一份完整 lm_head 权重的 fp32 副本——248k 词表下是 2.03 GB,而且每个 chunk 重建一次、每次梯度重计算再重建一次。修法就是把投影留在 bf16 张量核上。\n\n官方微基准(单 chunk,256 token × 词表 248,320 × hidden 2048,1×H100,bf16 前向+反向):23.37 ms \u002F 5.99 GB 降到 3.86 ms \u002F 3.03 GB,6.0 倍。落到端到端(tokens\u002Fs\u002FGPU,每步 16,384 token):Qwen3-8B 全参微调在 2×H100 FSDP2 上从 3554 提到 6009(1.69×),LoRA r16 从 4531 到 7125(1.57×),Qwen3-30B-A3B(MoE)1.20×,gemma-3-270m 1.32×。蒸馏 trainer 原本每个 chunk 要付两次这笔开销(学生一份、教师一份),同样受益;在 accelerate 混合精度下数值位级一致。\n\n## 百万 token 训练,从 trick 变成官方文档路径\n\nv1.13.0 同步发布了长上下文训练指南和一个可跑的示例:单节点 8×H100,每步训练一整本书长度的序列。实测配置是 Qwen3-8B、单条序列 1,048,576 token,380 秒\u002F步,每卡 56.2 GB(bf16,`loss_type=\"chunked_nll\"`)。指南按你实际会撞到的顺序给出杠杆:`chunked_nll`、梯度检查点 offload、YaRN RoPE;约束也写得很直白——只支持 full attention、不支持 packing,checkpointing offload 需要 transformers ≥ 5.16。\n\n## PPOTrainer:2020 年的起点,2026 年删掉\n\nBreaking 部分才是有历史感的:`PPOTrainer`、`PPOConfig` 和 value-head 封装整体移除。这条代码 2020-03-28 随仓库首个 commit(`dfb6a580`)落地,当时包还叫 `lm_ppo`——官方更新日志的原话是,它是 TRL 里最老的东西、原初代码库的最后一块。移除理由也列得清楚:一年多没有功能维护、全部 trainer 里唯一从未对齐输入格式的那个(还在收 tokenized `input_ids`)、记录在案的使用量趋近于零,还持续吸引自动 bug 猎手对没人运行的代码提真实报告。`from trl import PPOTrainer` 在 v1.10 就已经失效,真正的 PPO 用户本来就钉在旧版本上;`create_reference_model` 保留(BCO、A2PO、Online DPO 还在用)。\n\n其他值得扫一眼的:Liger-Kernel 的 fused linear DPO\u002FKTO\u002FGRPO\u002FJSD 损失被 vendor 进 `trl.losses`(与 Liger v0.8.2 位级一致,`use_liger_kernel` 开关行为不变);vLLM 支持升到 0.28.0、弃掉 0.19.0;依赖下限抬到 peft ≥ 0.13.0、deepspeed ≥ 0.18.6。\n\n这件事的启示:训练栈里真正的浪费,往往不在算法,而在\"默认就这么写了很多年\"的数值习惯。21.6% 的 kernel 时间,一个 dtype 决策就拿回来了——你的训练脚本里,是不是也藏着一份没人看过第二眼的 fp32 lm_head?\n\n完整更新日志见 [GitHub Release](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Freleases\u002Ftag\u002Fv1.13.0)。\n","https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Freleases\u002Ftag\u002Fv1.13.0","24d5c6c5-6573-4180-a1fd-f1459842d1af",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"445dab7e-3bfc-4a4e-9b87-3f1fd5817927","en","TRL v1.13.0: dtype fix recovers 21.6% kernel time; PPO retires","TRL v1.13.0: fp32 upcast fix recovers 21.6% kernel time (Qwen3-8B throughput 1.69x), adds 1M-token training guide, removes 2020-era PPOTrainer.","TRL, Hugging Face's post-training library, shipped v1.13.0 this week, and the most consequential change in the log is a fix that is almost embarrassing in hindsight: the default chunked loss was upcasting its lm_head projection to fp32 before the matrix multiply, pushing the heaviest GEMM off the tensor cores onto the SIMT path. In an 8×H100 profile of `trl sft` on Qwen3.6-35B-A3B, those two fp32 SIMT GEMMs accounted for 21.6% of all GPU kernel time.\n\n## One dtype decision buys back a fifth of kernel time\n\nThe mechanism is simple: both operands of `h.float() @ w.float().t()` were already bf16, so the upcast bought no information. It did, however, materialize an fp32 copy of the entire lm_head weight — 2.03 GB for a 248k vocabulary — rebuilt for every chunk and again on every gradient-checkpoint recompute. The fix keeps the projection on bf16 tensor cores.\n\nOfficial micro-benchmark (one chunk, 256 tokens × vocab 248,320 × hidden 2048, 1×H100, bf16, fwd+bwd): 23.37 ms \u002F 5.99 GB before, 3.86 ms \u002F 3.03 GB after — 6.0×. End to end (tokens\u002Fs\u002FGPU, 16,384 tokens per step): Qwen3-8B full fine-tuning on 2×H100 FSDP2 goes 3554 → 6009 (1.69×); LoRA r16 goes 4531 → 7125 (1.57×); Qwen3-30B-A3B (MoE) 1.20×; gemma-3-270m 1.32×. Distillation trainers paid the fp32 cost twice per chunk (student and teacher) and benefit equally; under accelerate mixed precision the numerics are bit-identical.\n\n## Million-token training becomes a documented path\n\nv1.13.0 also ships a long-context guide and a runnable example that trains a book-length sequence per step on a single 8×H100 node. The measured config: Qwen3-8B at 1,048,576 tokens per sequence, 380 s\u002Fstep, 56.2 GB per GPU (bf16, `loss_type=\"chunked_nll\"`). The guide documents the levers in the order you actually hit them — `chunked_nll`, gradient-checkpointing offload, YaRN RoPE — and states the constraints plainly: full attention only, no packing, and checkpointing offload needs transformers ≥ 5.16.\n\n## PPOTrainer: the 2020 starting point, deleted in 2026\n\nThe breaking change is the one with history: `PPOTrainer`, `PPOConfig`, and the value-head wrappers are gone. That code landed on 2020-03-28 in the repository's first commit (`dfb6a580`), when the package was still called `lm_ppo` — in the release notes' own words, the oldest thing in TRL and the last piece of the original codebase. The stated reasons: no feature work in over a year, the only trainer never aligned on input format (it still took tokenized `input_ids`), near-zero recorded usage, and a magnet for automated bug hunters filing real reports against code nobody runs. `from trl import PPOTrainer` already stopped working in v1.10, so anyone actually running PPO was pinned to an older TRL anyway; `create_reference_model` stays (BCO, A2PO and Online DPO use it).\n\nAlso worth a glance: the fused linear DPO\u002FKTO\u002FGRPO\u002FJSD losses from Liger-Kernel are vendored into `trl.losses` (bitwise identical to Liger v0.8.2, `use_liger_kernel` behavior unchanged); vLLM support moves to 0.28.0 and drops 0.19.0; dependency floors rise to peft ≥ 0.13.0 and deepspeed ≥ 0.18.6.\n\nThe takeaway: in modern training stacks, the real waste is often not in the algorithm but in numerical habits that survived because \"that's how it was always done\". 21.6% of kernel time, recovered by one dtype decision — how many unexamined fp32 lm_head projections are hiding in your training script?\n\nFull changelog: [GitHub Release](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Freleases\u002Ftag\u002Fv1.13.0).\n","trl-v1-13-tensor-core-lmhead-ppotrainer","2026-09-12T21:05:00Z","2026-09-12T21:09:17.674642Z","2026-09-12T21:09:17.674659Z",true,"agent",47,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"4c4a2a9e-f69b-4985-bd42-97ab2ef4e2ac","Spark-X2.5-4B 开源:4B 跑 1M 上下文,22 项基准打 9B 级 Qwen3.5","spark-x2-5-4b-apache-open-source","2026-09-16T01:30:00+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"936c14f5-53ec-4c86-8a5c-bb2def8b7dff","清华团队 5090 美元从零预训练 2B 模型,完整配方开源","puro-2b-open-pretraining-recipe","2026-08-31T17:10:00+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"6fa1bc74-c98e-476b-bc4c-9ae057105ffb","ParaTempo:免训练并行推理,延迟最高降 32%、token 省三成","paratempo-temporal-confidence-parallel-reasoning","2026-08-24T17:20:00+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"491f4904-c854-4925-b3e3-e34b8afd5e50","KDA+MLA 混合栈下沉到 1.3B 激活:Ling-3.0-tiny 把 MoE 端侧化,INT4 跑出 115 tok\u002Fs","ling-3-tiny-kda-mla-edge-deployment","2026-08-18T00:00:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"259d91b2-ed6b-4af8-8f2a-f759b84cc617","蚂蚁 Ling-3.0 Flash：124B\u002F5.1B MoE 的 Agent 生产级模型","inclusionai-ling-3-flash-hybrid-linear-moe-agent","2026-08-14T08:00:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"c71b8ee7-9487-4c78-89fd-30bb0368b99e","DeepSeek V4 Flash：284B\u002F13B MoE，成本比 Luna 低 60%","deepseek-v4-flash-0731-intelligence-index-50","2026-08-05T03:00:00+00:00"]