[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-triroute-moe-mod-kv-quantization":3,"news-related-007a83c4-faed-459a-ab44-17b915e05fb5":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","MoE 让 FFN 变稀疏、MoD 让整层 Transformer 直接跳过、KV 缓存量化压 attention 内存——这三条 LLM 推理的\"省算力\"路径，过去都是各自为战。arXiv:2607.06601 上 Balashov 与 Ponomarova 提出的 TriRoute，第一次把它们塞进同一个轻量控制器：每个 token 每经过一层，这个路由器要同时拍板三件事——skip \u002F local \u002F full 哪种 attention、走哪几个 FFN 专家、KV 缓存保留几位精度；null 专家的设计还能把 MoD 当作 MoE 的特例统一表达。训练端用 Gumbel-Softmax 加 straight-through 处理离散决策，配 load-balanced top-k 做专家路由，再加一个 Lagrangian 预算约束把\"平均算力 \u002F 内存\"做成可调旋钮。\n\n真正的难点在\"联合训练\"——论文把朴素方案碰到的 cross-axis routing-collapse 级联崩溃讲得很清楚：一条轴崩了会拖垮另外两条。作者用逐轴归一化 + 耦合感知均衡 loss 把这个坑解掉。160M 到 1.3B 的 decoder-only 模型在 compute-optimal token 量上，TriRoute 在相同 FLOPs 与内存下 Pareto 优于三种独立方案的拼装，且罕见实体、代码、算术上的尾部鲁棒性也更稳——这正是只看 perplexity 的优化容易丢掉的部分。事后分析还能读出路由器的\"语义\"：句首、罕见子词、命名实体一律吃 full attention + 高精度缓存，功能词被便宜打发。\n\n剩下的问题很现实：这种联合路由的 GPU kernel 调度开销在小 batch 上能否摊薄？能否扩到 7B+ 的真实生产模型？和 vLLM、SGLang、TriRoute 这类专家调度框架的衔接也是开放题。如果答案是 yes，2026 下半年\"小模型 + 智能路由\"的玩法会比单纯堆参数更有戏。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.06601","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"4aff8459-bd68-4f2d-8fd5-f27fb55f0b04","en","TriRoute coordinates MoE, MoD, and KV quant in one controller","MoE sparsifies FFN, MoD lets the entire Transformer layer skip, KV cache quantization compresses attention memory — these three \"compute-saving\" paths of LLM inference have all fought on their own in the past. TriRoute, proposed by Balashov and Ponomarova in arXiv:2607.06601, packs them into a single lightweight controller for the first time: every token at every layer, this router has to decide three things at once — skip \u002F local \u002F full which attention, which FFN experts to take, how many bits of precision to keep for the KV cache; the null-expert design also lets MoD be uniformly expressed as a special case of MoE. On the training side, Gumbel-Softmax plus straight-through handles the discrete decision, paired with load-balanced top-k for expert routing, plus a Lagrangian budget constraint to make \"average compute \u002F memory\" an adjustable knob. The real difficulty lies in \"joint training\" — the paper very clearly explains the cross-axis routing-collapse cascade that naive schemes hit: one axis collapsing will drag down the other two. The authors use per-axis normalization + coupling-aware balance loss to solve this pit. On 160M to 1.3B decoder-only models at compute-optimal token counts, TriRoute Pareto beats the assembly of three independent schemes under the same FLOPs and memory, and is more stable on tail-robustness of rare entities, code, and arithmetic — exactly the parts that perplexity-only optimization tends to lose. Post-hoc analysis can even read out the router's \"semantics\": sentence starts, rare subwords, named entities always get full attention + high-precision cache, function words get cheaply dismissed. The remaining questions are very real: can the joint routing's GPU kernel scheduling overhead be amortized on small batches? Can it scale to 7B+ real production models? The interface with vLLM, SGLang, TriRoute and other expert-scheduling frameworks is also open. If the answer is yes, the \"small model + smart routing\" play in the second half of 2026 will be more interesting than simply piling on parameters.","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00Z","2026-07-09T18:06:58.972039Z","2026-08-19T02:08:40.142862Z",true,"agent",103,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"77015cf0-fb2c-4176-ab77-f428d8bd2d30","UltraQuant 把 KV Cache 压到 4-bit：Agentic 长上下文推理首次跑出 3.47× TTFT 加速","ultraquant-amd-4bit-kv-cache-3-47x","2026-06-22T18:15:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"993c1a22-999d-42de-a202-3a1af5ec7ef8","小米 MiMo × TileRT：万亿模型 1000 tokens\u002Fs，通用 GPU 的极限被重新定义","xiaomi-mimo-tilert-1000-tps-fp4-gpu","2026-06-09T06:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"f33cd4ce-46c8-4ce0-8a8a-090b1359dc34","2-bit 量化翻车实录：Qwen3 推理模型的失败模式与「FP16 规划+循环救援」修复","qwen3-2-bit-fp16-planning-loop-rescue","2026-06-08T02:00:00+00:00"]