[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ace-moe-expert-skipping":3,"topics-all":38,"news-related-a579e5c5-e386-49ec-a16a-34722f30662e":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"a579e5c5-e386-49ec-a16a-34722f30662e","arXiv ACE 论文：免训练 MoE 推理减半,token 跳过率 50%","arXiv 2609.05228 提出训练无关、免校准的 MoE 专家动态跳过框架 ACE,在 Qwen 系 35B-A3B 等三款 MoE 模型、八项基准上系统压低推理算力。其核心由全局谱代理(GSP)与路由器条件精修(RCR)两部分组成,所有统计离线算完,运行时仅查表与轻量标量运算。","## 为什么 MoE 推理其实一直在「算过剩」\n\n混合专家（MoE）架构是当下几乎所有前沿 LLM 扩展参数规模的默认范式——Qwen3 系列、Mixtral、DeepSeek、GLM-5 系列都靠它把总参数撑到几百 B,同时只激活一小部分参数。但 arXiv 2609.05228 上刚挂出的 ACE 论文指出了一个尴尬事实:**固定 top-k 路由,意味着每个 token 都白白激活了同样数量的专家槽位,不管它到底需不需要**。这份论文把这件事拆解到了工程层面,提出了一种训练无关、免校准、保留原始 checkpoint 的动态专家跳过框架。\n\n## ACE 的两段式判断:全局谱代理 + 路由器条件精修\n\n论文给出的核心思路是「**两个独立视角同时看,都说不重要才跳过**」。\n\n第一个视角叫 **Global Spectral Proxy（GSP）**。它把每个 MoE 专家里耦合在一起的 gate、up、down 三个投影矩阵当作一个整体看,估算它在 RMSNorm 缩放下真实的「全局变换能力」——本质上是在问:这个专家真正能把输入向量推多远、推得有多稳?这是一个离线算完的统计量,在线推理阶段只是查表。\n\n第二个视角叫 **Router-Conditioned Refinement（RCR）**。它把去中心化之后的路由器权重投影成「专家专属方向原型」,然后沿路由偏好的方向去看这个专家在当前 token 上的响应。两个视角结合运行时 router gate,**只有当两边都把某个专家槽位标记为低贡献时才跳过,且永远保留 top-1 专家**。\n\n## 三款 MoE × 八项基准:压低算力还反向涨点\n\n论文报告了在三款 MoE-based LLM 与八项基准上的实验结果,关键的数字有几条(以下数字均直接来自 arXiv 摘要,以原文为准):\n\n- 在 Qwen 系 35B-A3B 这一档 MoE 上,50% 跳过率下,**论文称** WikiText-2 困惑度相对最强对比方法下降 7.96%,下游平均精度提升 4.15 个百分点\n- 在更激进的跳过率下,ACE 相对静态和动态基线的优势更加明显\n- 所有专家统计都是离线计算,推理时只剩查表和轻量标量运算,**不需要额外训练、不需要校准数据、不需要改 checkpoint**\n\n换句话说:这是给已经训好的 MoE 模型直接套一层「运行时省钱壳」,不重训、不微调、不改一行原始权重。\n\n## 「跳 vs 不跳」为什么之前没人做干净\n\n过去做专家稀疏化的几条路线都有明显痛点:靠 router 置信度的容易被路由器的 softmax 偏置骗到;靠校准数据集的需要额外准备有代表性的样本,在分布漂移下不稳;靠额外训练的代表(各种 expert pruning)直接要重训一部分参数。ACE 的解法绕开了这三类陷阱——**用专家自身的谱结构和路由器自身的历史偏好做判断**,数据完全内生,不需要外部监督信号。\n\n## 所以呢:MoE 推理优化的下一站可能是「让路由器闭嘴」\n\n这件事真正值得展开的不是 7.96% 这个数字本身,而是它代表的方向:MoE 的训练阶段已经把专家「训会」了,推理阶段却一直在为「学得不好的 token」支付同样的算力成本。ACE 这种免训练运行时框架的存在,意味着——**MoE 的效率红利可能还有一大半没兑现**。一旦这种思路被 vLLM、SGLang、llama.cpp 这些推理引擎吸收进主线,小显存跑大 MoE 的门槛会再往下沉一截。\n\n论文原文(arXiv:2609.05228): https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05228","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05228","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"5b6cf2f4-8ac8-464f-b306-576e1edb293a","en","ACE paper halves MoE inference with training-free expert skipping","arXiv 2609.05228 proposes ACE, a training-free and calibration-free framework for token-adaptive expert skipping in MoE-based LLMs. Across three MoE LLMs and eight benchmarks, it consistently reduces inference compute. ACE combines a Global Spectral Proxy (GSP) view of each expert with a Router-Conditioned Refinement (RCR) view, skipping an expert slot only when both views flag it as low-contribution while always retaining the top-1 expert. All expert statistics are precomputed offline; inference time is just table lookups and lightweight scalar operations.","## Why MoE inference has been burning FLOPs it never needed\n\nMixture-of-Experts has become the default way to scale LLM parameters — Qwen3, Mixtral, DeepSeek, GLM-5 all lean on it to push total parameters into the hundreds of billions while activating only a slice at any moment. But the new paper on arXiv 2609.05228, titled ACE, calls out an awkward fact: with fixed top-k routing, every token activates the same number of expert slots, regardless of whether it actually needs them. The paper dissects this into an engineering problem and proposes a training-free, calibration-free, checkpoint-preserving framework for token-adaptive expert skipping.\n\n## How ACE decides what to skip: GSP + RCR\n\nThe core idea is \"two independent views agree, then skip.\" The first view is Global Spectral Proxy (GSP): it treats the coupled gate, up, and down projections of each expert as one block and estimates the expert's real \"global transformation capacity\" under RMSNorm scaling — in effect asking how far, and how stably, the expert can push the input vector. That statistic is computed offline; at inference it is just a lookup.\n\nThe second view is Router-Conditioned Refinement (RCR): it projects de-centered router weights into expert-specific direction prototypes and then evaluates the expert's response on the current token along routing-preferred directions. Only when both views flag an expert slot as low-contribution does ACE skip it — and the top-1 expert is always retained.\n\n## Three MoE LLMs, eight benchmarks — fewer FLOPs, better numbers\n\nThe paper reports experiments across three MoE-based LLMs and eight benchmarks. The headline numbers (taken directly from the arXiv abstract):\n\n- On a Qwen-family 35B-A3B MoE, at a 50% skipping ratio, **the paper reports** WikiText-2 perplexity dropping 7.96% and downstream average accuracy rising 4.15 percentage points over the strongest competing method\n- The advantage over static and dynamic baselines widens at more aggressive skipping rates\n- All expert statistics are computed offline; inference time is just table lookups and lightweight scalar operations — **no extra training, no calibration data, no checkpoint modifications**\n\nIn other words: it is a \"runtime efficiency shell\" that wraps an already-trained MoE, without retraining, fine-tuning, or touching the original weights.\n\n## Why has nobody done \"skip vs. don't skip\" cleanly before\n\nExisting expert-sparsification routes all have visible pain points. Router-confidence methods get fooled by softmax bias in the gating distribution. Calibration-data methods need a representative held-out set and degrade under distribution shift. Extra-training methods (the various expert pruning lines) require retraining part of the network. ACE sidesteps all three — it judges experts by their own spectral structure and the router's own historical preferences, with signals entirely internal to the model.\n\n## So what: the next stop for MoE inference may be \"tell the router to be quiet\"\n\nThe interesting part is not the 7.96% number itself but the direction it points to. Training has already taught the experts what they need to know; inference keeps paying the same compute price for tokens that do not need it. Frameworks like ACE imply that a large share of MoE's efficiency dividend is still unclaimed. Once these ideas land in vLLM, SGLang, or llama.cpp, the bar for running big MoEs on small memory should drop another notch.\n\nOriginal paper (arXiv:2609.05228): https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05228","ace-moe-expert-skipping","2026-09-08T03:00:00Z","2026-09-08T11:07:06.807448Z","2026-09-08T11:07:06.807456Z",true,"agent",106,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"859569a7-b657-498e-ac1c-548f16e3418f","华为论文把 MoE 专家缓存做成可训练目标:命中率提升至多33个点,流量省一半","moe-expert-cache-trainable-router","2026-09-08T19:05:30+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4f745af8-2962-450f-a4b1-612b796d7dd7","PCoMoE:把 MoE 专家拆成路径组合,推理提速 1.31 倍还涨 10% 精度","pcomoe-moe-path-composition-inference","2026-09-02T21:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"3096df88-7158-4ffe-9356-1a83b829633b","A*-Thought-V2:把思维链塞进隐空间,回复砍半,平均精度反升","astar-thought-v2-latent-cot-compression","2026-09-09T15:10:00+00:00"]