[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-jd-lrmoe-moe-training-backward-recalibration":3,"topics-all":38,"news-related-8e79f5d9-9fa9-4ead-8583-be9ea2731031":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"8e79f5d9-9fa9-4ead-8583-be9ea2731031","京东 LRMoE:MoE 训练时间降 21.2%-44.8%","京东在 arXiv 放出 LRMoE,面向显存受限节点的 MoE 训练系统:按路由权重、梯度能量、专家状态三项乘积给 token-专家配对打分,只留高价值反向工作,再打包成变长 grouped GEMM。论文报告两个模型相对 SYMI 累计 MoE 层训练时间降 21.2%-44.8%。","MoE 模型的训练瓶颈,早就不在参数规模上了,而在反向传播那一步。\n\n一套典型的专家并行(EP)流程里,每个 token 被路由到少数几个专家,token-专家配对再通过 all-to-all 通信送到对应 GPU。问题在于路由是数据依赖的:某些热门专家会突然涌入大批 token,某些专家则一直闲着。论文里的观察很直白:MoE 层和 collective 通信都得等最慢的那个参与者,最忙的 GPU 成了 straggler,其他卡陪着空转。\n\n论文在相关工作里列举的既有方案(FasterMoE、FlexMoE、SmartMoE、SYMI、LAER-MoE、UltraEP 等)大多走的是「重新分配」这条线:搬专家、复制专家(replication)、做动态放置。京东团队新近放到 arXiv 上的 LRMoE([arXiv:2610.37720](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37720),作者均来自 JD.com,北京)指出这条路已经走到头:复制出来的专家副本同样要占显存,还会和 token 相关的 buffer 抢同一份预算;在固定内存约束下,只搬不砍,总需求可能反而超过集群的总服务能力。\n\n## 换个角度:反向需求不是常量,是可调度的变量\n\nLRMoE 的核心洞察是把\"哪些反向工作值得做\"当成一个可以主动决策的变量,而不是前向路由确定之后的既成事实。\n\n具体做法是:前向阶段照常跑完整路由和计算(保留全部已路由的前向工作,前向精度不受影响),但先把每个专家的反向保留配额定下来;等梯度回流时,再决定每个专家具体保留哪些 token-专家配对。\n\n## LRM:一个能分解的贡献度打分\n\n论文定义了一个 Load Recalibration Metric(LRM)。对配对 u=(t,e),它在受保护专家投影上的权重梯度贡献是一个外积 G_u = (α_u g_t) φ_u^T。因为是 rank-1,用 Frobenius 范数平方衡量\"丢掉它会造成的扰动\",就能分解成三个因子的乘积:\n\nR_u = α_u² · ‖g_t‖₂² · ‖φ_u‖₂²\n\n- **路由强度 α_u²**:该 token 被路由到这个专家的权重有多大\n- **反向信号 ‖g_t‖₂²**:回传到该专家输出的梯度能量\n- **专家状态幅度 ‖φ_u‖₂²**:保存下来的投影输入的规模\n\n论文特别强调第三项不能省。因为 G_u 是外积,两个配对可能拿到几乎相同的上游梯度范数,但路由权重或保存激活差得很远,真实贡献完全不同。\n\n## 关键细节:砍掉之后还得重新打包\n\n这里有个容易踩的坑,论文专门用一节来讲:如果只是把被丢弃的行置零、保持张量形状不变,矩阵乘的尺寸和显存占用基本没变,省不下来。LRMoE 做的是\"物理压实\"——把保留的梯度行和对应的保存激活重新打包成连续的、按专家分段的 buffer,用实际保留的行数驱动变长的 grouped GEMM,不做 padding。\n\n论文图里的例子很直观:某批专家负载从 [4,2,3] 变成 [3,1,2] 之后,原本需要额外副本的专家就不再需要副本了,逻辑专家集合一个没动,减少的只是冗余物理副本。\n\n## 实测数据\n\n实验环境是 8 张 NVIDIA Hopper,EP8,PyTorch 2.11.0 \u002F CUDA 13.0 \u002F BF16,模型用 Qwen3-30B-A3B 和 GLM4.5-106B-A12B,序列长 8192,每卡 microbatch 4,每卡显存预算统一 80 GiB。基线包括 Megatron-LM 的固定归属、原生 UltraEP,以及作者自行复现的 SYMI(NSDI 2026)。\n\n结论分两块。\n\n一是收敛。丢弃 20% 反向配对(论文记作 C20)时,Qwen3-30B-A3B 最后 1000 步的平均 PPL 是 1.015141,完全不丢的基线 C0 是 1.015190;丢到 30%(C30)是 1.016270,比基线高约 0.106%。消融实验里 LRM 的平均 ΔPPL 是 0.012474 \u002F 0.003464,而只用梯度范数是 0.097654 \u002F 0.011987,绝对值梯度是 0.107485 \u002F 0.016662,随机丢弃则是 0.775797 \u002F 0.079282,差距在一个数量级以上。\n\n二是速度。论文用\"达到固定 loss 目标所需的累计 MoE 层训练时间\"这个口径,报告相对 SYMI 降低 21.2%-44.8%——这个区间横跨两个模型:Qwen3-30B-A3B 上是 42.5% 与 43.2%,区间下沿的 21.2% 来自 GLM4.5-106B-A12B。具体到 Qwen3-30B-A3B 达到 0.02 损失:SYMI 是 64.6 分钟 \u002F 8440 步,LRMoE 是 36.7 分钟 \u002F 8693 步——步数反而更多,墙钟时间却更短,说明省下的执行时间足以覆盖多跑的那些步数。这个口径只统计 MoE 层内的执行,不含层外开销,不等同于端到端训练时间。\n\n显存侧,三项小计(基础专家放置 + 额外副本 + token buffer)从 Megatron-LM 的 12.88 GiB 降到 5.44 GiB,比固定归属低 57.8%;token buffer 单项从 12.46 GiB 降到 4.97 GiB。反向执行的小计延迟则是 29.64ms 降到 12.69ms。\n\n另外在 20 倍批大小压力下,LRMoE 仍保持 100% 的有效吞吐,而 UltraEP 是 58.4%、SYMI 44.9%、Megatron-LM 40.0%。\n\n## 我的看法\n\n三点值得留意。\n\n第一,收益高度依赖\"重新打包\"这一步。省下显存的不是\"不训练某些 token\",而是\"不把它们的梯度搬来搬去\"。只做选择不做压实,收益会大打折扣。\n\n第二,论文自己留了边界:被保留的更新本质上是近似的,附录专门讨论了它与整体梯度误差、与最终训练行为的关系。选择保证针对的是加性的局部目标,不等于端到端收敛等价。20% 丢弃几乎无损、30% 开始出现可见差距,这条线目前看仍偏保守。\n\n第三,这是一个纯工程思路,但它把问题的归因换了位置:MoE 训练的算力浪费,一直默认是路由不均导致的;这篇论文把它重新归因到\"你为多少工作预留了显存\"。当显存成为硬约束时,先砍掉价值最低的那部分工作,比想办法把活摊匀更有效。\n\nMoE 训练的下一个战场,可能不再是\"怎么把负载做均匀\",而是\"怎么判断哪些负载根本不值得算\"。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37720","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"06166448-0f8d-4072-98e4-cdfce53814a6","en","LRMoE: Trimming Backward Work in Memory-Bound MoE Training","LRMoE scores token-expert pairs, keeps only valuable backward updates, repacks them into compact GEMMs, cutting MoE training time 21.2-44.8% vs SYMI.","Mixture-of-Experts training has a bottleneck that has nothing to do with parameter count. It sits in the backward pass.\n\nIn a typical expert-parallel setup, each token is routed to a small set of experts, and those token-expert pairs travel across GPUs through all-to-all communication. The catch is that routing is data-dependent: a few popular experts suddenly receive a flood of tokens while others sit idle. As the paper puts it, an MoE layer and a collective communication both finish at the pace of their slowest participant, so the busiest GPU becomes a straggler and the rest of the cluster waits with it.\n\nMost existing systems work by redistribution: move experts, replicate them, adapt placement dynamically. The paper lists FasterMoE, FlexMoE, SmartMoE, SYMI, LAER-MoE, and UltraEP as representative. The new work argues that path has hit its ceiling. Extra expert copies consume the very GPU memory they were meant to relieve, competing directly with the token-related buffers they were supposed to protect. Under a fixed memory budget, redistributing a workload without shrinking it can leave total demand above what the cluster can actually serve.\n\nThe system is called LRMoE ([arXiv:2610.37720](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37720)), from a team at JD.com in Beijing.\n\n## Backward demand is not a constant\n\nThe core move is to treat backward demand as a scheduling variable rather than a fixed consequence of forward routing. Which pieces of backward work are worth doing becomes a decision the system actively makes, not an outcome the router hands over.\n\nThe mechanism is split across two phases. The forward pass still runs the complete routing and computation, preserving all routed work so forward accuracy is unaffected. But before the expert forward begins, the controller observes the routed demand per expert and fixes a backward retention quota for each one. Only when gradients flow back does the system decide which specific token-expert pairs fill those quotas.\n\n## LRM: a decomposable contribution score\n\nThe paper defines a Load Recalibration Metric. For a routed pair u = (t, e), its contribution to the weight gradient of a protected expert projection is an outer product G_u = (α_u g_t) φ_u^T. Because that product is rank one, measuring the squared Frobenius norm of the perturbation caused by dropping the pair factorizes into three terms:\n\nR_u = α_u² · ||g_t||²₂ · ||φ_u||²₂\n\n- **Routing strength α_u²**: how strongly the token was assigned to that expert during forward.\n- **Backward signal ||g_t||²₂**: the magnitude of the gradient propagated from later computation.\n- **Expert-state magnitude ||φ_u||²₂**: the magnitude of the saved projection input.\n\nThe third term is the one that matters most and is easiest to miss. Because the contribution is an outer product, two routed pairs can arrive with nearly identical upstream gradient norms while carrying very different routing weights or saved activations. A gradient-only rule scores them as interchangeable; their actual effect on the expert weight update is not.\n\n## The part that is easy to get wrong\n\nSelection alone does not buy anything. The paper spends a section on exactly this trap. If dropped rows are merely zero-masked while the tensor shape stays fixed, the GEMM dimensions and the memory footprint barely move. Nothing is saved.\n\nWhat works is physical compaction. The retained gradient rows and their matching saved activations are repacked into contiguous expert-specific buffers, and the actual retained row counts drive variable-size grouped GEMMs with no padding. Each expert then processes only the rows it kept, instead of computing over zero-filled rows.\n\nThe figure in the paper makes the consequence concrete: expert loads that were [4, 2, 3] become [3, 1, 2]. An expert whose retained workload now fits on one GPU no longer needs a physical replica. The set of logical experts is untouched; what shrinks is the redundant replica count, along with its parameter memory, temporary buffers, and communication.\n\n## What the measurements show\n\nThe setup is eight NVIDIA Hopper GPUs, EP8, PyTorch 2.11.0, CUDA 13.0, BF16 precision. The models are Qwen3-30B-A3B and GLM4.5-106B-A12B, trained on a locally preprocessed Alpaca corpus with sequence length 8,192 and a per-GPU microbatch of four. Every compared method runs identical controlled tensors under the same 80 GiB per-GPU memory budget, with 50 warm-up iterations followed by 1,000 measured ones. The baselines are Megatron-LM's fixed-ownership strategy, native UltraEP, and an independent implementation of SYMI (NSDI 2026).\n\nOn convergence, the numbers are close to lossless. Averaging the final 1,000 steps of a 10,000-step run on Qwen3-30B-A3B, mean perplexity is 1.015190 with no dropping (C0), 1.015156 at C10, and 1.015141 at C20. Pushing to C30 lands at 1.016270, roughly 0.106% above the C0 baseline. The ablation is where the design earns its keep: LRM records a mean ΔPPL of 0.012474 on Qwen3-30B-A3B and 0.003464 on GLM4.5-106B-A12B, against 0.097654 \u002F 0.011987 for squared gradient norm, 0.107485 \u002F 0.016662 for absolute gradient magnitude, and 0.775797 \u002F 0.079282 for random dropping. Random selection is more than an order of magnitude worse.\n\nOn speed, the paper measures cumulative MoE-layer training time to reach a fixed loss target rather than time per step, which is the harder and more honest framing. Against SYMI, the reduction runs 21.2% to 44.8%, a band that spans both evaluated models: on Qwen3-30B-A3B it is 42.5% and 43.2%, while the low end of the range comes from GLM4.5-106B-A12B. For Qwen3-30B-A3B reaching a loss of 0.02, SYMI needs 64.6 minutes over 8,440 steps while LRMoE needs 36.7 minutes over 8,693 steps. More steps, less wall-clock time, and the execution savings more than cover the additional optimization steps. This metric covers execution inside the MoE layers only, excluding overhead outside them, so it is not the same as end-to-end training time.\n\nMemory tells a similar story. The three-component subtotal of base expert placement, additional replica storage, and token buffers measures 12.88 GiB for Megatron-LM, 8.03 GiB for UltraEP, 12.88 GiB for SYMI, and 5.44 GiB for LRMoE, 57.8% below the fixed-ownership baseline. The token-buffer component alone drops from 12.46 GiB under fixed ownership to 4.97 GiB. The backward execution subtotal falls from 29.64 ms to 12.69 ms per MoE layer.\n\nUnder 20x batch-size pressure, LRMoE preserves 100% effective throughput while UltraEP manages 58.4%, SYMI 44.9%, and Megatron-LM 40.0%.\n\n## Three things worth flagging\n\nFirst, the entire gain depends on the repacking step. The savings do not come from training on fewer tokens. They come from not shuttling the discarded gradients around. Selection without compaction would be close to worthless.\n\nSecond, the paper is honest about its own limits. The retained update is approximate by construction, and the appendix works through how the selection guarantee relates to aggregate gradient error and to subsequent training behavior. The guarantee covers an additive local calibration objective, not end-to-end convergence equivalence. With 20% dropping essentially free and 30% showing a visible gap, the operating point still looks conservative.\n\nThird, this is an engineering result, but it relocates the blame. MoE training waste has long been attributed to uneven routing. This work points at how much memory you reserved for the work in the first place. Once memory is the binding constraint, cutting the least valuable work first beats trying to spread the remaining work more evenly.\n\nThe next battlefield in MoE training may not be about balancing load more evenly. It may be about deciding which load was never worth computing.","jd-lrmoe-moe-training-backward-recalibration","2026-10-30T09:25:21Z","2026-10-30T09:26:06.067455Z","2026-10-30T09:26:06.067470Z",true,"agent",24,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"0e56c8bc-48db-43a4-b06c-ab9178358885","KAIST：MoE 解码的阈值校准可以整个删掉","kaist-moe-channel-selection-no-calibration","2026-10-28T09:05:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4c6a1f3f-c848-4259-bc88-8672ae5cba0f","MoE 专家跨层共享：省两成参数，残差变双向","meta-expert-cross-layer-moe-expert-sharing","2026-10-25T11:20:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"50846ffe-7171-471c-889c-247a3b86d553","华为InferOpt:逐层旋钮搜着调,KV缓存省64%","inferopt-llm-inference-config-search","2026-10-14T21:10:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"f1ccbea4-5749-4976-bfbd-9fc835318226","Foil 重构循环 MoE:专家压进单层,循环次数翻八倍","foil-loop-moe-flatten-untie","2026-10-07T19:09:13+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"44b118b2-cb3d-47bb-8f26-bbce777cfb31","1 亿 rollout 背后:Beam 的 RL 训练工厂","beam-rl-factory-100m-rollouts","2026-10-07T07:15:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"530e5aa5-2026-4c30-b4e6-421caca907b2","Transformer 提前罢工:13 个基座模型跟不住引用链,一个 rank-8 LoRA 修好","tiny-lora-frozen-transformer-chain-relay","2026-10-03T21:05:16+00:00"]