[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-moe-pruning-english-calibration-language-erosion":3,"topics-all":38,"news-related-9260ffaf-bab9-4444-8852-b2689df55ab3":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"9260ffaf-bab9-4444-8852-b2689df55ab3","MoE剪枝英语校准:立陶宛语掉21.4分,英语仅1.7","专家剪枝是 MoE 省显存的常规操作,校准集默认英语。新论文对四款开源 MoE 做 48 组对照:25% 剪枝下,英语校准只让英语阅读理解掉 1.7 分,立陶宛语却掉 21.4 分;换立陶宛语校准,立陶宛语找回 25.5 分,英语反涨 2.5 分。英语成绩单完全看不见这场暗跌,末层路由集中是根源。","MoE 开源模型省显存的通行做法,是把路由器很少用到的专家整块剪掉,剪掉一半专家能省下 44.3%-47.5% 的参数。但剪谁不剪谁,取决于一份小小的校准集——而绝大多数团队的校准集默认英语。新提交到 arXiv 的 41 页论文(编号 2611.07654)用 48 组受控实验给出回答:英语校准剪掉的,主要是小语种的能力,而且英语成绩单几乎看不出任何动静。\n\n## 48 组受控网格:四款主流开源 MoE\n\n实验对象是四款 26-35B 总参、3-4B 激活的开源 MoE:Qwen3-30B-A3B、Qwen3.6-35B-A3B、Gemma-4-26B-A4B、GLM-4.7-Flash,恰好是小集群部署常见的一档。剪枝方法是 REAP——按路由器权重和专家输出大小给每个专家打分,分数最低的整层剔除;每层剪掉 25% 或 50% 的路由专家。校准集三选一:纯英语、纯立陶宛语、五五混合;语料域二选一:维基百科或欧洲议会平行语料。4 模型 × 2 语料域 × 3 校准语言 × 2 剪枝比例,共 48 个剪枝模型,再加 4 个未剪枝基线,全部在阅读理解(Belebele)、双向翻译(FLORES-200)和维基百科语言建模上做英语\u002F立陶宛语双语评测。\n\n立陶宛语是块好的试金石:它只占 Common Crawl 的 0.17%,英语占 42.6%;同一句话,立陶宛语要多花 1.7-2.1 倍 token;七个名词格,九个带变音符的字母,犯错空间极小。\n\n## 核心结果:伤害完全不对称\n\n25% 剪枝、英语校准下,英语阅读理解中位数只掉 1.7 分,立陶宛语掉 21.4 分。极端个案:Qwen3-30B 配欧洲议会英语校准,英语保持 93.3%(仅比基线低 1.0),立陶宛语从 89.9% 直坠 22.3%——四选一瞎猜也有 25%。三套基准的立-英退化差距全部同向:阅读理解 +23.5 分,译入立陶宛语 +12.8 chrF++,语言建模退化 +50.0 个百分点。\n\n校准集换成立陶宛语,剧本立刻反转:立陶宛语阅读理解平均找回 25.5 分,译入立陶宛语找回 28.2 chrF++,翻译成绩全面回到未剪枝模型 1.1 chrF++ 以内;英语非但没受损,阅读理解反而平均高 2.5 分。在这组网格里,校准语言一项就解释了立-英差距方差的 39%-70%,超过模型、剪枝比例、语料域任何一个因素——它才是主导变量。\n\n模型之间也有温差:专家数 256 的 Qwen3.6 在 50% 剪枝下平均只掉 11.5 分,其余三款要掉 16.7-66.4 分;同一个 Qwen3-30B,欧洲议会英语校准把立陶宛语打到 22.3%,维基百科英语校准下还能剩 67.4%——校准语料域的影响同样肉眼可见。\n\n## 为什么:路由集中在两头\n\n论文给出的机制解释是路由结构。网络中段,两种语言共用大部分专家;到了首尾两层,立陶宛语开始走自己的路,而末层它把 55%-72% 的专家选择压在仅仅 8 个专家上,英语对应的数字是 19%-39%。英语校准平均剪掉这 8 个专家里的 0.86-2.88 个,立陶宛语校准至多 0.71 个。被英语校准剔除的专家,此前承载着 20.2% 的立陶宛语路由,却只承载 13.6% 的英语路由——对英语无关痛痒,对立陶宛语是半条命脉。\n\n## 剪到一半:失效模式清单\n\n剪枝比例翻到 50%,损失放大 2.4-6.9 倍,连立陶宛语校准也兜不住:其余三款模型仍要掉 6-15 分,Gemma 4 则无论哪种校准,立陶宛语全面崩到随机水平,英语 6 组里也有 4 组如此。失效形态非常具体:GLM-4.7-Flash 在 25% 剪枝时就有 27%-36% 的译入立陶宛语输出刹不住车、撞上 256 token 上限,欧洲议会英语校准的 50% 配置达到 94%;50% 英语校准下,59.4% 的输出英语词多过立陶宛语词,立陶宛语校准只有 10.9%,未剪枝模型为零;模型开始造词,32.6% 的立陶宛语形词在任何参考译文里都不存在,立陶宛语校准时仅 6.2%;还有 948 条输出只吐出半个 UTF-8 字节、变音符直接乱码,其中 88% 来自 GLM-4.7-Flash。\n\n## 所以呢\n\n这篇论文里分量十足的一句不在结论,在评测含义那节:25% 英语校准的 8 组配置里,5 组英语成绩保持在基线 2 分以内,同时立陶宛语暗跌 19.3-67.6 分。只看英语榜单,这种损伤永远不可见。作者留了三条实操建议:用目标语言或混合语料做校准;保守剪枝;上线前用目标语言测生成,盯住跑飞、串语言、重复循环、字母塌缩这些退化信号。对中文场景,结论不能直接外推——论文没测中文,中文的语料占比与路由结构和立陶宛语都不同——但方法学教训是普适的:压缩报告里不写校准语言,这份报告就只对校准语言负责。\n\n论文原文:[arXiv 2611.07654](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.07654)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.07654","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"89ccdc15-a832-42fb-a714-d397959503fc","en","English Calibration Erodes Lithuanian in Pruned MoE Models","Expert-pruning MoE models with English calibration costs Lithuanian 21.4 points vs 1.7 English; LT calibration restores 25.5, English gains 2.5.","The standard way to shrink an open MoE model is expert pruning: delete the experts the router rarely uses, and cutting half of them removes 44.3%-47.5% of the parameters. But which experts get deleted is decided by a small calibration set — and for most teams that set defaults to English. A newly submitted 41-page arXiv paper (2611.07654) answers the question with 48 controlled experiments: English calibration mostly carves away the capabilities of lower-resource languages, while the English scorecard stays almost motionless.\n\n## A controlled grid of 48 pruned models\n\nThe subjects are four open MoE models with 26-35B total and 3-4B active parameters: Qwen3-30B-A3B, Qwen3.6-35B-A3B, Gemma-4-26B-A4B, and GLM-4.7-Flash — the tier commonly deployed on small clusters. The pruning method is REAP: score each expert by router weight and output magnitude, then delete the lowest-scoring experts layer by layer, removing 25% or 50% of routed experts per layer. The calibration set is one of three languages — English only, Lithuanian only, or a 50\u002F50 mix — over one of two domains, Wikipedia or the parallel Europarl corpus. Four models, two domains, three calibration languages, and two pruning ratios yield 48 pruned models, plus 4 unpruned baselines, all evaluated in both English and Lithuanian on reading comprehension (Belebele), bidirectional translation (FLORES-200), and Wikipedia language modelling.\n\nLithuanian is a good litmus test: it accounts for 0.17% of Common Crawl against 42.6% for English; the same sentence costs 1.7-2.1 times more tokens in Lithuanian; and with seven noun cases and nine diacritic letters, the room for error is tiny.\n\n## The core finding: the damage is wildly asymmetric\n\nAt 25% pruning with English calibration, English reading comprehension loses a median of 1.7 points while Lithuanian loses 21.4. The extreme case: Qwen3-30B calibrated on Europarl English keeps 93.3% English accuracy (only 1.0 below baseline) while Lithuanian crashes from 89.9% to 22.3% — random guessing on four options gives 25%. The Lithuanian-minus-English degradation gap points the same way on all three benchmarks: +23.5 points on reading comprehension, +12.8 chrF++ on translation into Lithuanian, and +50.0 percentage points of relative language-modelling degradation.\n\nSwitch the calibration set to Lithuanian and the script flips: Lithuanian reading comprehension recovers 25.5 points on average, translation into Lithuanian recovers 28.2 chrF++, and translations land back within 1.1 chrF++ of the unpruned models; English is not hurt at all — comprehension is on average 2.5 points better. Within this grid, the calibration language alone explains 39%-70% of the variance in the gap, more than the model, the pruning ratio, or the calibration domain — it is the dominant variable.\n\nThere is also spread between models: Qwen3.6, with 256 experts, loses only 11.5 points on average at 50% pruning while the other three lose 16.7-66.4; and the same Qwen3-30B that crashes to 22.3% under Europarl-English calibration still holds 67.4% under Wikipedia-English calibration, so the calibration domain swings results on its own.\n\n## Why: routing concentrates at the two ends\n\nThe paper's mechanistic account is routing structure. Through the middle of the network the two languages share most experts; toward the first and last layers Lithuanian takes its own routes, and in the last layers it presses 55%-72% of its expert selections onto just 8 experts, against 19%-39% for English. English calibration removes on average 0.86-2.88 of those 8 experts per layer; Lithuanian calibration removes at most 0.71. The experts deleted under English calibration had carried 20.2% of Lithuanian routing but only 13.6% of English routing — painless for English, half a lifeline for Lithuanian.\n\n## Prune to 50%: a checklist of failure modes\n\nDoubling the pruning ratio multiplies losses by 2.4-6.9, and even Lithuanian calibration cannot hold the line: the other three models still lose 6-15 points, while Gemma 4 collapses to chance in Lithuanian under every calibration set and in English in 4 of 6. The failure shapes are concrete: at 25% pruning, 27%-36% of GLM-4.7-Flash translations into Lithuanian run away and hit the 256-token limit, reaching 94% in the Europarl-English 50% configuration; at 50% English calibration, 59.4% of outputs contain more English than Lithuanian words, against 10.9% under Lithuanian calibration and none for unpruned models; the models invent words, with 32.6% of Lithuanian-looking words absent from any reference output, versus 6.2% under Lithuanian calibration; and 948 outputs emit only half a UTF-8 byte sequence, breaking diacritics into invalid characters, 88% of them from GLM-4.7-Flash.\n\n## So what\n\nThe heaviest sentence of the paper is not in the conclusions but in the implications for evaluation: 5 of the 8 English-calibrated configurations at 25% stay within 2 points of their unpruned English accuracy while silently losing 19.3-67.6 points in Lithuanian. On an English-only leaderboard this damage is forever invisible. The authors leave three practical recommendations: calibrate on the target language or a mix; prune conservatively; and before deployment, test generation in the target language, watching for runaway length, language switching, repetition loops, and letter collapse. For Chinese deployments the numbers do not transfer directly — the paper did not test Chinese, whose corpus share and routing structure differ from Lithuanian — but the methodological lesson is universal: a compression report that does not state its calibration language is only accountable to that one language.\n\nFull paper: [arXiv 2611.07654](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.07654)","moe-pruning-english-calibration-language-erosion","2026-11-07T13:30:00Z","2026-11-07T13:17:41.608904Z","2026-11-07T13:17:41.608915Z",true,"agent",199,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"f828f611-b57a-4855-9246-3609c2495faa","老师折进自己:2.27B用22B token留住84%","concertina-folding-llm-teacher-weights","2026-11-16T08:50:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"25191dae-d51d-4fdd-a23c-3f1a46563c8a","MoE剪枝别看专家看神经元:留一半,显存省36-43%","moe-neuron-specialization-rampr","2026-11-11T15:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"228ec943-ecfa-4010-b74d-6b8bf791328e","字节用SAE拆MoE:Java和前端代码分属不同专家","sae-interprets-moe-qwen3-expert-specialization","2026-11-07T21:20:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"18fa7eb9-44e7-4722-ad95-52fc7d795434","Qwen4 开训:阿里把大模型推到 5-10 万亿参数","qwen4-training-5-10-trillion-params","2026-10-03T05:00:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"d056f67b-7e0d-4e44-8d39-e31ea50deeae","Bonsai 2 27B 三元压缩:Qwen3.8 压到 5.9 GB,benchmark 留存 98.2%","bonsai-2-27b-ternary-qwen3-8-compression","2026-09-17T15:47:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"d941056b-c2e7-42e5-965a-a982c20b1169","Qwen3.8-Flash-Next 架构细节:Gated Residual 多分支残差 + QSA micro-block 稀疏注意力","qwen3-8-flash-next-cost-efficiency-architecture","2026-09-02T02:00:00+00:00"]