The standard way to shrink an open MoE model is expert pruning: delete the experts the router rarely uses, and cutting half of them removes 44.3%-47.5% of the parameters. But which experts get deleted is decided by a small calibration set — and for most teams that set defaults to English. A newly submitted 41-page arXiv paper (2611.07654) answers the question with 48 controlled experiments: English calibration mostly carves away the capabilities of lower-resource languages, while the English scorecard stays almost motionless.

A controlled grid of 48 pruned models

The subjects are four open MoE models with 26-35B total and 3-4B active parameters: Qwen3-30B-A3B, Qwen3.6-35B-A3B, Gemma-4-26B-A4B, and GLM-4.7-Flash — the tier commonly deployed on small clusters. The pruning method is REAP: score each expert by router weight and output magnitude, then delete the lowest-scoring experts layer by layer, removing 25% or 50% of routed experts per layer. The calibration set is one of three languages — English only, Lithuanian only, or a 50/50 mix — over one of two domains, Wikipedia or the parallel Europarl corpus. Four models, two domains, three calibration languages, and two pruning ratios yield 48 pruned models, plus 4 unpruned baselines, all evaluated in both English and Lithuanian on reading comprehension (Belebele), bidirectional translation (FLORES-200), and Wikipedia language modelling.

Lithuanian is a good litmus test: it accounts for 0.17% of Common Crawl against 42.6% for English; the same sentence costs 1.7-2.1 times more tokens in Lithuanian; and with seven noun cases and nine diacritic letters, the room for error is tiny.

The core finding: the damage is wildly asymmetric

At 25% pruning with English calibration, English reading comprehension loses a median of 1.7 points while Lithuanian loses 21.4. The extreme case: Qwen3-30B calibrated on Europarl English keeps 93.3% English accuracy (only 1.0 below baseline) while Lithuanian crashes from 89.9% to 22.3% — random guessing on four options gives 25%. The Lithuanian-minus-English degradation gap points the same way on all three benchmarks: +23.5 points on reading comprehension, +12.8 chrF++ on translation into Lithuanian, and +50.0 percentage points of relative language-modelling degradation.

Switch the calibration set to Lithuanian and the script flips: Lithuanian reading comprehension recovers 25.5 points on average, translation into Lithuanian recovers 28.2 chrF++, and translations land back within 1.1 chrF++ of the unpruned models; English is not hurt at all — comprehension is on average 2.5 points better. Within this grid, the calibration language alone explains 39%-70% of the variance in the gap, more than the model, the pruning ratio, or the calibration domain — it is the dominant variable.

There is also spread between models: Qwen3.6, with 256 experts, loses only 11.5 points on average at 50% pruning while the other three lose 16.7-66.4; and the same Qwen3-30B that crashes to 22.3% under Europarl-English calibration still holds 67.4% under Wikipedia-English calibration, so the calibration domain swings results on its own.

Why: routing concentrates at the two ends

The paper's mechanistic account is routing structure. Through the middle of the network the two languages share most experts; toward the first and last layers Lithuanian takes its own routes, and in the last layers it presses 55%-72% of its expert selections onto just 8 experts, against 19%-39% for English. English calibration removes on average 0.86-2.88 of those 8 experts per layer; Lithuanian calibration removes at most 0.71. The experts deleted under English calibration had carried 20.2% of Lithuanian routing but only 13.6% of English routing — painless for English, half a lifeline for Lithuanian.

Prune to 50%: a checklist of failure modes

Doubling the pruning ratio multiplies losses by 2.4-6.9, and even Lithuanian calibration cannot hold the line: the other three models still lose 6-15 points, while Gemma 4 collapses to chance in Lithuanian under every calibration set and in English in 4 of 6. The failure shapes are concrete: at 25% pruning, 27%-36% of GLM-4.7-Flash translations into Lithuanian run away and hit the 256-token limit, reaching 94% in the Europarl-English 50% configuration; at 50% English calibration, 59.4% of outputs contain more English than Lithuanian words, against 10.9% under Lithuanian calibration and none for unpruned models; the models invent words, with 32.6% of Lithuanian-looking words absent from any reference output, versus 6.2% under Lithuanian calibration; and 948 outputs emit only half a UTF-8 byte sequence, breaking diacritics into invalid characters, 88% of them from GLM-4.7-Flash.

So what

The heaviest sentence of the paper is not in the conclusions but in the implications for evaluation: 5 of the 8 English-calibrated configurations at 25% stay within 2 points of their unpruned English accuracy while silently losing 19.3-67.6 points in Lithuanian. On an English-only leaderboard this damage is forever invisible. The authors leave three practical recommendations: calibrate on the target language or a mix; prune conservatively; and before deployment, test generation in the target language, watching for runaway length, language switching, repetition loops, and letter collapse. For Chinese deployments the numbers do not transfer directly — the paper did not test Chinese, whose corpus share and routing structure differ from Lithuanian — but the methodological lesson is universal: a compression report that does not state its calibration language is only accountable to that one language.

Full paper: arXiv 2611.07654