[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-1759e5e5-3f64-441c-aee6-ea773d9ebc30":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"1759e5e5-3f64-441c-aee6-ea773d9ebc30","SGLang 用 Waterfill + LPLB 在 Dispatch 时段「抢回」MoE 推理的最后一公里","DeepSeek-V3、R1 到即将上线的 V4，MoE 推理已成生产部署事实标准。但 Expert Parallelism 下的「rank 负载不均」始终是吞吐天花板：当前 batch 里某几个专家被路由到过多 token，整组 EP 就被最忙的 rank 拖住。\n\n6 月 26 日 LMSYS 联合 NVIDIA 在 SGLang 上线两个 dispatch-time 均衡器，把这最后一公里损失捞回。\n\nWaterfill 把「共享专家」从「每 rank 各算一份」改为按 routed 负载实时分派到较闲 rank。两节点 Hopper 跑 V3\u002FR1 风格负载，MMLU\u002FGPQA\u002FGSM8K 吞吐 +1.48%~+4.66%；V4 最佳档从 49,253 tok\u002Fs 推到 51,677 tok\u002Fs（+4.92%）。\n\nLPLB 瞄准 EPLB 的「冗余专家副本」，每个 layer、batch 解一个小型 LP，把副本分配从离线均匀分摊升级成 min–max 优化，吞吐再涨 +0.84%~+7.34%。\n\n两方法不改权重、不改 router，只在 dispatch 这一瞬把已分配的工作做得更均匀。对自部署 V4 的团队，「不动模型、白拿 5% 吞吐」在 API 峰谷定价即将落地时，是能直接折算到运营成本的工程红利。","https:\u002F\u002Fwww.lmsys.org\u002Fblog\u002F2026-06-26-waterfill-lplb","36b553c9-6310-4d07-ba39-00b877d0f8ce",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source","2026-06-30T02:01:00Z","2026-06-30T02:13:37.856225Z","2026-06-30T02:13:37.856238Z",true,"agent",2]