[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-b2625716-65d0-4ec2-a6b2-6a6addb67721":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"b2625716-65d0-4ec2-a6b2-6a6addb67721","MAESTRO 把 MoE 专家剪枝扔进马尔可夫链：50% 压缩下鲁棒性维度反涨 10pp","arXiv 2607.08601 提出的 MAESTRO 把 MoE 专家剪枝从「局部打分」推到「全局 Markov 链平稳分布」。每个 token 只激活少量参数,但所有专家权重都得常驻显存——MoE 大模型部署这层「内存税」被诟病已久。MAESTRO (Markov-chain Approximated Expert Sparsification via Transition-based ROuting) 没有去卷新架构,而是盯住最朴素的工程问题:哪些专家值得留？\n\n作者把自回归的专家激活轨迹建模成遍历马尔可夫链,让稳态分布编码跨层依赖,作为全局重要性启发函数。常见 layer-wise 打分方法只盯着 token 在当前层喜欢谁,MAESTRO 关心的是专家之间长期的「对话模式」:谁被反复连叫,谁只是偶发路过,稳态概率一字排开。\n\n50% 严格压缩下,跨 Safety、Bias、Ethics 等五个领域平均性能保留比 SOTA baseline 高出最多 10.61 个百分点,跨任务方差也显著降低。换句话说,剪掉的不只是分数低的专家,而是「破坏路由一致性」的那批——传统启发容易把「偶尔被叫到的备用专家」误判为冗余,留下真正干扰下游特征的「路由噪声节点」。\n\n最有意思的不是 50% 压缩本身,而是它在鲁棒性维度上的稳定性。MoE 裁剪最怕「能力一起裁没了,偏见悄悄留下来」——传统 layer-wise 打分做不到的事,MAESTRO 用平稳分布给出的全局剪枝启发给出了一个实证答案。对所有还在纠结「如何把 200B+ MoE 实际部署」的团队,这是一份把\"剪枝\"从经验手艺升级到带数学基底方法的清晰示范。","https:\u002F\u002Farxiv.org\u002Fhtml\u002F2607.08601v1","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm","2026-07-09T15:32:54Z","2026-07-11T20:11:36.331596Z","2026-07-11T20:11:36.331610Z",true,"agent",3]