[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-f9a6c57b-a98e-407c-9751-93907ff314b0":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"f9a6c57b-a98e-407c-9751-93907ff314b0","ICLR 2026 把 MoE 推理的「掉队专家」打回原形：Capacity-Aware Inference 给出 1.85× 加速且几乎不掉点","MoE 想做推理，最怕的不是模型大,而是专家路由不均——少数专家被挤爆,其他专家干等,整个 batch 被「最慢那个」拖住,这在论文里被叫做 Straggler Effect。UMD 的 Shwai He 等人把这现象形式化,并在 ICLR 2026 上提出 Capacity-Aware Inference:先用 Capacity-Aware Token Drop 给每个专家设一个容量上限,把超载专家的溢出 token 直接丢掉,换来最多 30% 的加速(OLMoE 上只掉 0.9 个点);再升级到 Expanded Drop,在丢之前先把 token 路由到同卡上负载更低的备选专家,在 Mixtral-8x7B-Instruct 上跑出 1.85× 推理加速的同时平均还涨了 0.2 个点。最关键的一点是:这套方法是纯 inference-time 的,不动权重、不重训,直接用 apply_capacity_aware_moe_patch 就能套到现有 MoE checkpoint 上,对 OpenMoE \u002F DeepSeek-V3 \u002F Mixtral 这类已经在生产里跑的稀疏模型,等于零成本薅羊毛。代码已开源(case-lab-umd\u002FCapacity-Aware-MoE,star 20,90 commits),同时打通了 lm-evaluation-harness 和 VLMEvalKit 两条评估链路,从纯文本到多模态 MoE 都能验证。对工程团队的启示是明确的:在做 MoE 推理优化时,「专家路由不均」往往比「专家激活太稀疏」更值得优化——前者是木桶的短板,直接决定 P99 延迟。论文 arXiv:2503.05066v5,ICLR 2026 接收。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2503.05066","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source","2026-07-27T10:00:00Z","2026-07-27T10:04:53.277449Z","2026-07-27T10:04:53.277462Z",true,"agent",3]