[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-kaist-moe-channel-selection-no-calibration":3,"topics-all":38,"news-related-0e56c8bc-48db-43a4-b06c-ab9178358885":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"0e56c8bc-48db-43a4-b06c-ab9178358885","KAIST：MoE 解码的阈值校准可以整个删掉","KAIST 提出 MoE 稀疏解码的阈值不必离线校准。Prox 按 18% 预算校准后只读 12.9%，改用 top-k 后 43 组配对比较中 40 组持平或更优。其 reader 以三值 sketch 筛通道、低秩映射补回，在三个后训练 MoE 模型上用 17.8% 路由专家读取保留 98.1% 精度。","MoE 模型推理的瓶颈早就不在算力，而在显存带宽：每个 token 都要从显存里把路由到的专家权重搬一遍，小批量交互式解码时，这部分开销直接压过计算本身。近期提交到 arXiv 的论文《Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data》(arXiv:2610.37386) 冲着这个瓶颈来，但它真正挑战的不是\"怎么挑通道\"，而是稀疏推理里一个被默认接受的前提——阈值必须拿离线数据校准。\n\n论文由 Geunhyoung Kim 与 Sung Ju Hwang 署名，在 Hugging Face Daily Papers 榜单上挂在 KAIST AI 名下；正文 37 页、12 张图，代码以 Apache 2.0 协议开源。\n\n## 校准阈值锁不住读取量\n\n论文先拿最常见的做法开刀：channel selector 给每个通道打分，再用离线校准出来的阈值决定\"读哪些通道\"。问题在于阈值卡的是分数，实际读多少权重却取决于有多少通道的分数碰巧越线，而这个数量随输入变化。论文的测量是：Prox 按 18% 的读取预算校准，在他们的基准上实际平均只读 12.9%，最低到 9.9%；论文把其中约 1.7 个百分点归因于 Prox 已发表的预算分配规则本身。\n\n读取量按硬件实际收费的口径统计——GPU 按 32 字节一个 sector 收费，碰到就算整段。论文用 profiler 交叉验证过：Qwen3-Thinking 在 batch one 时的冷缓存 DRAM 读取是 18.3%，和他们自己统计的 17.8% 对得上。\n\n## top-k 换掉阈值\n\n论文的对照实验设计得很干净：同一个 selector、同一套打分，只换保留规则，一个用校准阈值，一个用 top-k（每个 token 只留分数最高的 k 个通道），在阈值原本校准的那档预算上比。43 组配对比较里，top-k 版本 30 组打平（差异在两个配对标准误内）、10 组领先、3 组落后，生成任务上的 23 组全部是持平或领先。\n\n更值得注意的是失败模式。Gemma 4 上，校准阈值在重度稀疏下会掉进重复循环，长推理时停不下来；同一批 selector 换成 top-k 就避开了。论文的解释是稀疏模型的重复一旦开始就很难自己停下来，精度不是平滑下滑，而是断崖式下跌。\n\n## 不用任何数据的 reader\n\n论文进一步做了一个只用模型自身权重的 reader：每个专家的 gate 与 up 投影保留一份旋转的三值 sketch（约占 BF16 checkpoint 的 7%），常驻 GPU，先用它给候选通道粗筛；top-k 用原生 BF16 权重精确算出；被跳过的通道则用 down 投影的低秩映射补回来，全程不需要校准数据。\n\n对比下来，各种校准版的零件反而是负收益：论文报告称，一个带校准的 recovery map 在 18 组比较里 0 组赢过他们的纯权重版；CATS 在 53% 和 67% 读取预算下，18 组里打平 16 组。\n\n## 数字与边界\n\n按论文自测：他们这个 reader 在三个后训练 MoE 模型上，用 17.8% 的路由专家读取量平均保留 98.1% 的全模型精度，最差的模型-基准组合也有 95.0%；同一张表里，Prox 打分配 top-k 在 18.5% 读取下平均 96.1%、最差 90.4%，而 Prox 按原论文方式校准后是平均 68.5%、最差 12.1%。Gemma 4 的 IFEval 上，全模型 93.5 分，Prox 配 top-k 在 13.5% 读取下 57.1 分，他们的 reader 在 12.8% 读取下 88.0 分。\n\n速度方面，论文报告 Triton 实现的 MoE block 比调优过的 vLLM 快 1.60 到 1.90 倍（batch one），更大 batch 下最高 2.69 倍；整模型解码最高快 1.88 倍，收益峰值落在 batch 8 或 16。\n\n模型是 Qwen3-30B-A3B-Thinking-2507、gemma-4-26B-A4B-it、Qwen3.5-35B-A3B，OLMoE-1B-7B-0924 用于似然任务；基准是 GPQA-Diamond、IFEval、MMLU-Pro；环境为 vLLM 0.29.0 + Triton 3.7.1 + CUDA 12.9。\n\n边界也得说清楚：所有数字都是论文自报，目前没有第三方复现，GitHub 仓库只有 3 次提交、3 个 star；更要命的是部署门槛——要跑这套稀疏运行时得给 vLLM 0.29.0 打两个 patch，仓库里就放了两个补丁文件，动的是 vLLM 自己的内部接口。\n\n## 所以呢\n\n这篇论文有意思的地方不在\"提速几倍\"，而在于它把一笔基础设施层的隐含债务摊到了台面上：稀疏 MoE 推理被默认要\"先准备一批校准数据\"，而这件事既贵又要持续维护——模型换一批、数据分布一变，阈值就可能对不上解码分布。用 top-k 替掉阈值不是新发明，但它把\"必须校准\"这个前提直接删掉了，对量化、稀疏注意力这些同样靠校准阈值的路线都适用。\n\n不过别急着改生产环境：两个 vLLM 补丁、3 个 star、没有第三方复现，短期内这更像一份值得跟踪的方法论，而不是明天就能上线的组件。真正的信号是——MoE 推理的下一个竞争点，可能不在模型本身，而在\"谁能把这些稀疏路径变成默认路径\"。[论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37386) \u002F [代码](https:\u002F\u002Fgithub.com\u002FAidenGeunGeun\u002Fcalibration-free-moe)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37386","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"fbe76761-3c8e-4b74-80c2-6aecd1b37d9b","en","KAIST: MoE Decoding Needs No Calibration Data","MoE decoding needs no calibration data: Prox thresholds set for 18% reads read only 12.9%, and top-k wins 40 of 43 tests at 98.1% accuracy.","The bottleneck in MoE inference stopped being arithmetic a long time ago. It is memory bandwidth: every token requires the routed expert weights to be pulled out of DRAM, and at small batch sizes during interactive decoding, that traffic swamps the actual math. A recent arXiv paper, \"Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data\" (arXiv:2610.37386), takes aim at that bottleneck — but what it really challenges is not *how* to pick channels. It challenges an assumption the sparse inference literature treats as settled: that the threshold you use to pick channels has to be calibrated on offline data.\n\nThe paper is by Geunhyoung Kim and Sung Ju Hwang, listed under KAIST AI on the Hugging Face Daily Papers leaderboard. It runs 37 pages and 12 figures, and the code is released under Apache 2.0.\n\n## Calibrated thresholds do not pin down how much you read\n\nThe paper starts with the common recipe: a channel selector scores every channel, and an offline-calibrated threshold decides which ones to read. The problem is that the threshold constrains a *score*, while how much weight actually gets read depends on how many channels happen to clear the bar — and that count shifts from input to input. The measurement: Prox, calibrated for an 18% read budget, actually reads only 12.9% on average across the paper's benchmarks, and as little as 9.9%. The paper attributes roughly 1.7 percentage points of that gap to Prox's own published rule for splitting the budget between inputs and channels.\n\nReads are counted the way hardware charges for them — GPUs bill in 32-byte sectors, so a sector touched at all is billed in full. The authors cross-check with a profiler: on Qwen3-Thinking at batch one, cold-cache DRAM reads measure 18.3%, matching the 17.8% they count.\n\n## Swapping the threshold for top-k\n\nThe comparison is cleanly designed. The same selector, the same scoring function, only the keep rule changes: one version uses the calibrated threshold, the other uses top-k (keeping the k highest-scoring channels per token), compared at the budget the threshold was originally calibrated for. Across 43 paired comparisons, the top-k version ties 30 of them (within two paired standard errors), leads 10, and trails 3 — and on generation tasks all 23 are ties or wins.\n\nThe failure mode matters more than the score. On Gemma 4, calibrated thresholds under heavy sparsity fall into repetition loops during long reasoning and never stop; the same selectors with top-k avoid that. The paper's explanation is that once a sparse model starts repeating, it rarely stops on its own, so accuracy drops as a cliff rather than a slope.\n\n## A reader that needs no data at all\n\nThe paper goes further with a reader built entirely from the model's own weights. For each expert, the gate and up projections keep a rotated ternary sketch of the weights (about 7% of the BF16 checkpoint), resident on the GPU, used to screen candidate channels cheaply; the top-k is computed exactly from the native BF16 weights; and the skipped channels are recovered through a low-rank map of the down projections. No step uses calibration data.\n\nThe calibrated alternatives turn out to be net negatives here. The paper reports that a calibrated recovery map beats their weight-only map in 0 of 18 comparisons, and that CATS ties in 16 of 18 comparisons at 53% and 67% read budgets.\n\n## Numbers and limits\n\nBy the paper's own measurements: across three post-trained MoE models, the reader retains 98.1% of full-model accuracy at 17.8% of routed-expert reads, and the worst model-benchmark pair still keeps 95.0%. In the same table, Prox score with top-k reaches 96.1% mean and 90.4% worst at 18.5% reads, while Prox as published and calibrated gets 68.5% mean and 12.1% worst. On Gemma 4's IFEval, the full model scores 93.5, Prox with top-k reaches 57.1 at 13.5% reads, and the paper's reader reaches 88.0 at 12.8% reads.\n\nOn speed, the paper reports Triton kernels running the MoE block 1.60 to 1.90 times faster than optimized vLLM at batch one, and up to 2.69 times at larger batch sizes; whole-model decoding is up to 1.88 times faster, with the gain peaking at batch size eight or sixteen.\n\nThe models are Qwen3-30B-A3B-Thinking-2507, gemma-4-26B-A4B-it, and Qwen3.5-35B-A3B, with OLMoE-1B-7B-0924 used for the likelihood tasks; the benchmarks are GPQA-Diamond, IFEval, and MMLU-Pro; the environment is vLLM 0.29.0 with Triton 3.7.1 on CUDA 12.9.\n\nThe limits deserve stating plainly. Every number above is self-reported by the paper, with no third-party reproduction yet; the GitHub repository has 3 commits and 3 stars. And the deployment bar is real: running this sparse runtime requires patching vLLM 0.29.0 with two patches, both shipped in the repository, and those patches touch vLLM's own internal interfaces.\n\n## So what\n\nWhat is interesting here is not \"how many times faster\" but the fact that the paper puts an infrastructure debt on the table. Sparse MoE inference silently assumes you first prepare a batch of calibration data, and that is both expensive and a permanent maintenance burden — swap in a different batch of models or shift the data distribution, and the thresholds stop matching the decoding distribution. Replacing thresholds with top-k is not itself new, but it deletes the \"you must calibrate\" premise outright, and that carries over to quantization and sparse attention, which lean on calibrated thresholds in the same way.\n\nStill, do not rewrite production on this basis: two vLLM patches, three stars, no third-party reproduction — for now this is a methodology worth tracking, not a component you can ship tomorrow. The real signal is that the next competitive frontier in MoE inference may not live in the model at all, but in who turns these sparse paths into the default path. [Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37386) \u002F [Code](https:\u002F\u002Fgithub.com\u002FAidenGeunGeun\u002Fcalibration-free-moe)","kaist-moe-channel-selection-no-calibration","2026-10-28T09:05:00Z","2026-10-28T09:12:32.078681Z","2026-10-28T09:12:32.078691Z",true,"agent",5,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"50846ffe-7171-471c-889c-247a3b86d553","华为InferOpt:逐层旋钮搜着调,KV缓存省64%","inferopt-llm-inference-config-search","2026-10-14T21:10:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"859569a7-b657-498e-ac1c-548f16e3418f","华为论文把 MoE 专家缓存做成可训练目标:命中率提升至多33个点,流量省一半","moe-expert-cache-trainable-router","2026-09-08T19:05:30+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"a579e5c5-e386-49ec-a16a-34722f30662e","arXiv ACE 论文：免训练 MoE 推理减半,token 跳过率 50%","ace-moe-expert-skipping","2026-09-08T03:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4f745af8-2962-450f-a4b1-612b796d7dd7","PCoMoE:把 MoE 专家拆成路径组合,推理提速 1.31 倍还涨 10% 精度","pcomoe-moe-path-composition-inference","2026-09-02T21:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"c2e41947-c219-4fbe-83f2-5dd9e4501e73","信号处理审计:推理链置信度评分其实量的是长度","token-confidence-chain-length-audit","2026-10-28T11:12:21+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"af6bfcff-1ffa-400d-ac93-cb6803ab232c","dSpark调度蒸馏:扩散LLM解码最多快10倍","dspark-schedule-distillation-diffusion-llm","2026-10-27T17:09:34+00:00"]