[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-inferopt-llm-inference-config-search":3,"topics-all":38,"news-related-50846ffe-7171-471c-889c-247a3b86d553":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"50846ffe-7171-471c-889c-247a3b86d553","华为InferOpt:逐层旋钮搜着调,KV缓存省64%","华为团队arXiv发布InferOpt,把LLM推理逐层KV保留率与MoE专家数建模为约束多目标优化,自动搜索质量-效率Pareto前沿。Qwen2.5-7B上16K缓存省64.4%,TPOT最多降48.5%;DeepSeek-V2-Lite砍43%路由对,质量仅掉0.59分。","LLM 推理系统里藏着一排没人愿意碰的旋钮:每一层保留多少 KV cache、每一层激活多少个专家。行业惯例是用统一策略或某个压缩论文给的启发式,拍一个\"全局一刀切\"的参数就上线。华为团队 10 月 3 日挂在 arXiv 的 InferOpt(论文 2610.04473)给出另一条路:别再发明第 N 个压缩启发式,把整件事当成约束多目标优化问题,交给搜索。\n\n## 把逐层旋钮变成搜索问题\n\n推理配置为什么难调?KV cache 压缩里,Qwen2.5-7B 每层一个保留率,构成 28 维连续空间;MoE 路由更狠,DeepSeek-V2-Lite 每层激活专家数取 1 到 6 的整数,是 26 维离散空间。而在十亿参数模型上,单个配置跑完整套评测就要数小时,穷举不现实。InferOpt 分三步走:先从数据集抽一个冻结的代理子集,所有候选配置只在同一批样本上打分;评估之前用闭式资源成本函数把超预算候选拦下,模型都不用跑;随可行解积累逐步收紧预算,最后把 Pareto 前沿代表配置放回全量数据复验。接入新机制只需定义变量域、资源成本和评估接口,搜索管线本身一行不改。\n\n## 数字:KV 省 64%,MoE 砍 43% 路由对\n\nKV 侧,在平均保留率 0.321 的激进预算下,16K 上下文缓存从 902.9 MiB 降到约 320 MiB,削减 64.4%;TPOT 从 19.11ms 降到 9.83ms,降幅 22.9%-48.5%。更值得注意的是对照实验:预算相同的条件下,搜出来的逐层分配比均匀分配质量高 3.21 到 6.13 分(7.3% 和 14.0%),而换打分器(attention 打分 vs key-L2 范数)只差 0.52 分——**分配比选择更重要**,这是论文里最反直觉的发现。MoE 侧,搜出的分段调度(前 9 层 k=6、中 9 层 k=3、后 8 层 k=1)砍掉 43.0% 的 token-专家路由对,TPOT 降 19.3%,质量距默认配置只差 0.59 分。对比 Random Search、NSGA-II、MOTPE 三个基线,InferOpt 在两个空间上都领先:KV 上超体积最高、保留率最低,MoE 上以最少的专家预算最接近未压缩参照。\n\n## 方法论迁移比数字更值钱\n\n单看数字,64% 和 43% 在压缩文献里不算离谱。真正的信号是论文自己点破的定位:它把 Ansor、Optuna 这类 ML 自动调参器的抽象,从底层 tensor 调度上移到了高层推理配置。推理优化的收益大头,可能不在又一个新算法,而在把已有旋钮的组合系统性搜一遍。对自建推理服务的团队,这意味着一套可复用的搜索层,KV、MoE、量化都能往里塞。当模型越来越贵在推理而不是训练,谁先把\"调参\"自动化,谁先把边际成本打下来。\n\n论文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.04473\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.04473","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"c3493694-c0a4-4f90-8940-038daf3b91c0","en","Huawei's InferOpt searches LLM inference knobs, cutting KV cache 64%","Huawei's InferOpt turns per-layer KV\u002FMoE knobs into multi-objective search: Qwen2.5-7B cache -64.4%, DeepSeek-V2-Lite routed pairs -43% at 0.59 points lost.","LLM serving systems hide a row of knobs nobody wants to touch: how much KV cache each layer retains, how many experts each MoE layer activates. The industry default is a uniform policy or a heuristic borrowed from some compression paper — one global setting, shipped. A Huawei team's InferOpt paper (arXiv 2610.04473, submitted October 3) argues for a different route: stop inventing the N+1th compression heuristic and treat inference configuration as what it really is — constrained multi-objective optimization.\n\n## Turning layer-wise knobs into a search problem\n\nWhy is this hard? In KV-cache compression, Qwen2.5-7B has one retention ratio per layer — a 28-dimensional continuous space. MoE routing is worse: DeepSeek-V2-Lite's per-layer activated-expert count ranges from 1 to 6, a 26-dimensional discrete space. Evaluating a single configuration on the full benchmark can take hours on a billion-parameter model, so exhaustive search is off the table. InferOpt runs three stages: it freezes a sampled proxy subset so every candidate is scored on identical examples; a closed-form resource-cost function rejects over-budget candidates before any model call; the budget tightens adaptively as feasible solutions accumulate, and Pareto-front representatives are re-validated on the full dataset. Plugging in a new mechanism requires only variable bounds, a resource-cost function, and an evaluation hook — the search pipeline itself is untouched.\n\n## The numbers: 64% less KV cache, 43% fewer routed pairs\n\nOn the KV side, at an aggressive mean-retention budget of 0.321, the 16K-context cache drops from 902.9 MiB to about 320 MiB — a 64.4% cut — while TPOT falls from 19.11ms to 9.83ms, a 22.9–48.5% reduction. The controlled comparison is the interesting part: at matched budgets, the searched layer-wise allocation beats uniform allocation by 3.21 to 6.13 quality points (7.3% and 14.0%), while swapping the selection scorer (attention-mass vs key-L2) moves quality by only 0.52 points. Allocation matters more than selection — the paper's most counterintuitive finding. On the MoE side, the searched segmented schedule (k=6 for the first 9 layers, k=3 for the middle 9, k=1 for the final 8) removes 43.0% of routed token–expert pairs and cuts TPOT by 19.3%, staying within 0.59 points of the default. Against Random Search, NSGA-II, and MOTPE, InferOpt leads on both spaces: best proxy hypervolume and lowest retention on KV, closest to the uncompressed reference at the lowest expert budget on MoE.\n\n## The method shift is worth more than the numbers\n\nIn isolation, 64% and 43% are not outrageous by compression-literature standards. The real signal is the framing the paper makes explicit: it extends the abstraction of ML auto-tuners like Ansor and Optuna from low-level tensor schedules up to high-level inference configuration. The biggest inference-optimization wins may come not from yet another algorithm, but from systematically searching the combinations of knobs we already have. For teams running their own serving stack, that means one reusable search layer that KV, MoE, and quantization can all plug into. As inference — not training — becomes where models cost the most, whoever automates the tuning first pushes marginal cost down first.\n\nPaper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.04473\n","inferopt-llm-inference-config-search","2026-10-14T21:10:00Z","2026-10-14T21:09:32.358799Z","2026-10-14T21:09:32.358813Z",true,"agent",33,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"859569a7-b657-498e-ac1c-548f16e3418f","华为论文把 MoE 专家缓存做成可训练目标:命中率提升至多33个点,流量省一半","moe-expert-cache-trainable-router","2026-09-08T19:05:30+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"a579e5c5-e386-49ec-a16a-34722f30662e","arXiv ACE 论文：免训练 MoE 推理减半,token 跳过率 50%","ace-moe-expert-skipping","2026-09-08T03:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"4f745af8-2962-450f-a4b1-612b796d7dd7","PCoMoE:把 MoE 专家拆成路径组合,推理提速 1.31 倍还涨 10% 精度","pcomoe-moe-path-composition-inference","2026-09-02T21:10:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ad9f5296-75bf-434c-a527-dc2703eb8c6a","Janas 引擎:80B 模型塞进 32GB 内存笔记本","janas-ssd-streaming-moe-inference","2026-10-14T13:12:19+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"dab2e487-cfba-4d75-a93f-eff133476348","剪枝校准对不上解码分布?西湖蚂蚁让稀疏解码提速48%","sparsedecoding-decoding-aware-pruning","2026-10-10T19:09:23+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"f1ccbea4-5749-4976-bfbd-9fc835318226","Foil 重构循环 MoE:专家压进单层,循环次数翻八倍","foil-loop-moe-flatten-untie","2026-10-07T19:09:13+00:00"]