arXiv 2606.19350 introduces CAP (Causal Attribution Pruning), a model pruning method that explicitly preserves the model's reasoning capability. The result: pruned models retain 95% of the original reasoning quality even at 50% sparsity, compared to 60-70% for standard pruning methods.
The problem: standard pruning methods (magnitude pruning, Wanda) optimize for "general loss," which doesn't capture reasoning-specific abilities. The result: pruned models lose reasoning disproportionately, with significant quality drops on math, code, and multi-step reasoning tasks.
The CAP fix: a "causal attribution" loss that measures each weight's contribution to reasoning tasks. The pruning is guided by this loss, so weights that are critical for reasoning are preserved, even if they have small magnitudes.
The technical details: CAP uses a "causal tracing" technique — for each reasoning task, the model is run with and without each weight, and the impact on the output is measured. The weights with the largest impact on reasoning tasks are preserved; the rest are pruned. The result is a sparse model with reasoning capability intact.
The benchmark: on a set of reasoning tasks (GSM8k, MATH, HumanEval), CAP-pruned models (50% sparsity) retain 95% of the original quality, while magnitude-pruned models retain only 67%. The CAP-pruned models are also faster — 1.8× speedup on H100.
The bigger takeaway: "task-aware pruning" is the right approach for reasoning models. The "one-size-fits-all" pruning assumption is breaking, and the future is "pruning that knows what the model is good at." For the industry, this means "efficient reasoning models" are now a real possibility — pruned models that retain reasoning capability at significantly lower cost.