The most expensive part of a speech LLM may not be the talking brain but the listening ears. XPeng's Omni team just open-sourced X-AuT, a compression framework whose answer is: shrink the ears, keep the brain frozen, and accuracy can even improve.
What Gets Pruned
X-AuT operates on the audio tower of speech LLMs like Qwen3-ASR-0.6B — the encoder that turns raw waveforms into embeddings the language model can consume. The team progressively pruned it from 18 Transformer blocks to 14: first removing layers 1 and 18 to get a 16-layer version, then removing layers 5 and 6 to reach 14. Audio-tower parameters dropped from 186M to 148M, a 20.7% cut. Throughout, the language-model backbone stays completely frozen; only LoRA adapters are added to attention projections, with the output embedding tied to lm_head.
The Numbers: A Two-Step Sweet Spot
Across ten public Chinese-English benchmarks, the results land at two practical operating points. The 16-layer variant pushes macro-average error from 5.61% down to 5.27%, a 6.1% relative reduction. The 14-layer variant sits at 5.75% error — just 2.5% relatively higher — in exchange for the 20.7% parameter cut. On inference efficiency, the 14-layer encoder cuts latency by 21.4% on the in-vehicle PPU and 11.4% on H800; end-to-end latency improves 4.7% and 2.6% respectively — modest, because the unpruned 28-layer text decoder dominates total inference time.
Why Progressive Beats One-Shot
Pruning 18 layers straight down to 14 pushes error to 6.73%; X-AuT's two-step approach with recovery training reaches 5.75% at the same depth — nearly a full point better. The recovery toolkit has four parts: behavior-driven layer probes, representation alignment, cross-scale distillation, and LoRA finetuning. The cross-scale distillation deserves a closer look: a frozen Qwen3-ASR-1.7B teacher supervises the 0.6B student through learned 2048-to-1024 projections, then is discarded after training. Under the matched recipe, the cross-scale teacher brings the student to 5.55% error versus 8.45% for same-scale self-distillation — a wide gap.
The On-Device Math
Behind these numbers is the reality of automotive compute: every millisecond of latency on a vehicle chip directly shapes user experience. On the data side, the team used a 280k-plus-hour source pool ranked into nine confidence classes by transcript consistency; all reported runs used class-1 data only. But some cold water is due: benchmark values are single-run best checkpoints with no repeated seeds or confidence intervals, as the paper itself states. The 14-layer model's largest degradation is 0.93 percentage points on Fleurs-en, and eight of ten benchmarks remain above baseline. The release scope is also limited — the repo ships the 14-layer checkpoint, a standalone inference script, and a minimal LoRA finetuning example, while the behavior-probe pipeline, distillation code, and the 280k-hour data pipeline stay private. The license is CC BY-NC 4.0, prohibiting commercial use.
For teams building on-device speech, the value here is not "yet another compression method" but a clearly mapped accuracy-latency frontier for the prune-the-audio-tower path: within two layers of progressive pruning, recovery training can erase or even reverse the accuracy loss; push deeper, and returns start racing risks. As voice interaction moves from phone assistants to vehicles, earbuds, and other compute-constrained scenes, this kind of careful slimming of the ears will land closer to production than blind parameter scaling.