[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-moe-expert-cache-trainable-router":3,"topics-all":38,"news-related-859569a7-b657-498e-ac1c-548f16e3418f":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"859569a7-b657-498e-ac1c-548f16e3418f","华为论文把 MoE 专家缓存做成可训练目标:命中率提升至多33个点,流量省一半","华为联合北航团队在 arXiv 发文,把 MoE 专家缓存管理从 serving 层启发式拉进模型侧后训练:辅助路由器只管显存驻留不碰 Top-K,实测命中率最多提升 33.34 个百分点,流量最高省 53.3%,新增参数仅占模型 0.083%。","MoE(混合专家)模型的卖点一直是\"参数很大、每个 token 只算一小部分\":路由器为每个 token 挑选少量专家参与计算,总容量上去了,单 token 计算量却没有同比例膨胀。但部署侧有个不太被大众讨论的尴尬——计算稀疏不等于显存友好。当专家总数超过单卡显存能装下的量,没被选中的专家只能躺在 CPU 内存或更慢的存储层,解码时被路由器点到的专家要临时搬回 GPU,权重搬运本身成了新瓶颈。\n\n华为联合北京航空航天大学等机构的研究团队,9 月 4 日提交到 arXiv 的论文(https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04895)选了一个不常见的攻击位置:与其在 serving 系统层用启发式规则猜\"哪些专家接下来会被用到\",不如把缓存管理本身做成模型侧的后训练目标。\n\n## 两个辅助路由器,不碰原生 Top-K\n\n方法分两层。Temporal Router(时间路由器)在每层专家计算完成后,预测同层专家在后续 token 上的复用概率,决定哪些专家留在 GPU 缓存里——它只管\"留谁\",不做主动预取,因此不产生任何前瞻传输。Spatio Router(空间路由器)则利用因果 Transformer 的执行顺序:目标层被访问之前,其\"因果前驱层\"的隐状态已经算好,可以用这个信号提前修正缓存内容。\n\n关键约束是:原生 MoE 路由器的 Top-K 专家选择规则在推理时完全保留,辅助路由器只管理\"谁驻留显存\",不改变\"谁执行计算\"。训练时用软化的 Top-B 隶属度代理构造缓存覆盖损失,与语言建模损失联合优化,让骨干网络和辅助路由器一起适配出缓存更友好的路由分布。\n\n## 实测:命中率与流量\n\n团队在 Qwen3 和 GPT-OSS 两个 MoE 骨干上,用 GSM8K、MATH、CommonsenseQA 三个推理基准评测(结果为五个种子的均值)。只做驻留更新的 Temporal Router 对比最强的经典替换策略(LRU\u002FLFU\u002FLRFU 一类),命中率在三个任务上分别提升 10.46、10.70、33.34 个百分点,每 token 解码流量从 1353\u002F1294\u002F1512 MB 降到 974\u002F906\u002F304 MB。加上空间路由的完整版,对比预取类基线中最强的 ProMoE,负载调整后命中率提升 1.15–18.03 个百分点,流量降低 4.6–53.3%。\n\n代价相当克制:完整版在 Qwen3 上新增 25.2M 推理参数,只占模型规模的 0.083%(GPT-OSS 上为 4.4M、0.021%),而 ProMoE 需要新增 96.0M。GPT-OSS 上的结果被论文如实描述为\"有竞争力但依赖任务\"。\n\n## 不是免费的午餐\n\n消融部分披露了几个真实代价。缓存损失权重 sw 调大,流量进一步下降,但精度开始掉:sw=0.1 时 Temporal Router 保持基线精度(GSM8K 85.44 不变),sw=2.0 时精度下降 10.84 个点。专家使用也向头部集中——熵从 3.96 降到 3.51,等效专家数从 53.41 缩到 34.54,专家并行的负载均衡可能受影响。作者还明确说明:Load 指标度量的是模拟的解码阶段流量,不等于端到端延迟;方法需要对全模型做后训练,不是即插即用的 serving 补丁。\n\n## 所以呢\n\n这篇论文的核心动作,是把 MoE 专家缓存从\"系统层启发式\"(MoE-Infinity、ProMoE、FineMoE 均属此类)拉进\"模型侧后训练\"的范畴。对显存受限的部署场景——比如 llama.cpp 这类想把大 MoE 塞进消费级设备的生态——这是值得跟踪的方向:缓存策略第一次成为可训练的优化目标,而不是靠 LRU 通用规则硬猜。代价也很清楚:全模型后训练加专家集中化副作用,离\"拿来就用\"还有距离。你愿意为省下一半权重流量,给模型做一次缓存感知的再训练吗?\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04895","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"1980a3ba-b82e-44df-9e22-878a3814df82","en","Huawei trains MoE cache routers: +33 hit points, traffic -53%","Huawei-Beihang arXiv paper makes MoE expert caching a trainable post-training objective: +33.34 hit-rate points, -53.3% traffic, 0.083% added params.","MoE (Mixture-of-Experts) models are sold on a simple pitch: huge total capacity, but each token computes only a small slice. A router picks a handful of experts per token, so total parameters grow without proportional per-token compute. The deployment-side catch is less discussed — compute sparsity does not equal memory friendliness. When the full expert set no longer fits in GPU memory, non-resident experts sit in host memory or slower storage, and every expert the router touches at decode time must be hauled back over the bus. Weight transfer becomes the new bottleneck.\n\nA team from Huawei, Beihang University and partner institutions posted a paper to arXiv on Sep 4 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04895) that picks an unusual place to attack this: instead of guessing \"which experts come next\" with serving-layer heuristics, make cache management itself a model-side post-training objective.\n\n## Two auxiliary routers, native Top-K untouched\n\nThe method has two stages. The Temporal Router runs after each MoE layer executes: it predicts same-layer expert reuse for future tokens and decides which experts stay in the GPU cache — retention only, no proactive loading, so it generates no speculative transfers. The Spatio Router exploits causal Transformer execution order: before a target layer is accessed, the hidden state of its causal predecessor is already computed, and that signal refines the cache ahead of the access.\n\nThe key constraint: the native MoE router's Top-K expert-selection rule is fully preserved at inference. Auxiliary routers manage residency, never which experts execute. During post-training, a softened Top-B membership surrogate builds a cache-coverage loss jointly optimized with the language-modeling loss, adapting the backbone and auxiliary routers together toward a cache-friendly routing distribution.\n\n## The numbers\n\nEvaluated on Qwen3 and GPT-OSS across GSM8K, MATH and CommonsenseQA (five-seed means), the update-only Temporal Router beats the strongest classical replacement policies by 10.46, 10.70 and 33.34 hit-rate points on the three tasks, while per-token decode traffic falls from 1353\u002F1294\u002F1512 MB to 974\u002F906\u002F304 MB. The full Spatio-Temporal Router improves adjusted hit rate by 1.15–18.03 points and cuts traffic by 4.6–53.3% versus ProMoE, the strongest prefetching baseline.\n\nOverhead is remarkably small: the full mode adds 25.2M inference-time parameters on Qwen3 — 0.083% of model size (4.4M, 0.021%, on GPT-OSS) — versus 96.0M for ProMoE. GPT-OSS results are described as \"competitive but task-dependent\", which the paper does not paper over.\n\n## No free lunch\n\nThe ablations disclose real costs. Raising the cache-loss weight sw pushes traffic down further but erodes accuracy: at sw=0.1 the Temporal Router keeps baseline accuracy (GSM8K 85.44 unchanged); at sw=2.0 accuracy drops 10.84 points. Expert usage also concentrates toward the head — entropy falls from 3.96 to 3.51 and effective experts shrink from 53.41 to 34.54, which may hurt expert-parallel load balance. The authors state plainly that Load measures simulated decode-stage traffic, not end-to-end latency, and that the method requires full-model post-training rather than being a drop-in serving patch.\n\n## So what\n\nThe core move here is pulling MoE expert caching from serving-layer heuristics (MoE-Infinity, ProMoE, FineMoE) into model-side post-training. For memory-constrained deployments — think llama.cpp-style projects pushing large MoE models onto consumer hardware — this is a direction worth tracking: cache policy becomes a trainable objective instead of LRU-style guessing. The caveats are equally clear: full-model post-training plus expert concentration side effects mean it is not plug-and-play yet. Would you retrain a model for cache awareness to cut your weight traffic in half?\n","moe-expert-cache-trainable-router","2026-09-08T19:05:30Z","2026-09-08T19:08:10.496728Z","2026-09-08T19:08:10.496737Z",true,"agent",122,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"a579e5c5-e386-49ec-a16a-34722f30662e","arXiv ACE 论文：免训练 MoE 推理减半,token 跳过率 50%","ace-moe-expert-skipping","2026-09-08T03:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4f745af8-2962-450f-a4b1-612b796d7dd7","PCoMoE:把 MoE 专家拆成路径组合,推理提速 1.31 倍还涨 10% 精度","pcomoe-moe-path-composition-inference","2026-09-02T21:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"3096df88-7158-4ffe-9356-1a83b829633b","A*-Thought-V2:把思维链塞进隐空间,回复砍半,平均精度反升","astar-thought-v2-latent-cot-compression","2026-09-09T15:10:00+00:00"]