[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-cap-causal-attribution-pruning-arc-61pct":3,"news-related-7cacddc6-fa02-4de9-84a2-c3320e225571":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"7cacddc6-fa02-4de9-84a2-c3320e225571","因果归因剪枝 CAP：让 LLM 推理能力不再随稀疏化而流失","arXiv 2606.19350（已被 ICLR 2026 Workshop on LLM Reasoning 接收）提出 Causal Attribution Pruning（CAP）。它针对的核心痛点是：传统 Wanda、SparseGPT 一类基于激活或幅值相关性的剪枝方法，长期主导 LLM 压缩，但相关性并不等于因果——少数 attention head 实际承担了关键的逻辑跳跃，而幅值标准对它们并不敏感。CAP 的做法是把每个 attention head 在 calibration 推理集上前向时 mask 掉，直接测量其对下游性能的下行幅度，即 \"interventional\" 因果分数，再把 head 级因果分数映射回 QKV\u002FO 投影矩阵，得到细粒度权重重要性。在 Llama-3-8B-Instruct 和 Mistral-7B-Instruct 上评估 GSM8K、StrategyQA、ARC-Challenge，覆盖 10%、20%、50% 三档稀疏度：20% 稀疏下，CAP 在 ARC-Challenge 上相对 Wanda 取得最高 61% 的相对精度增益；10–20% 区间，CAP 在大多数 (model, benchmark) 配置上都跑赢 Wanda。作者也坦诚 CAP 在 50% 稀疏度上被 MLP 层粗粒度因果归因拖累。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.19350","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"015eb272-b030-41fc-9cd6-d683b1dd6369","en","CAP: causal-attribute pruning keeps reasoning through sparsity","arXiv 2606.19350 introduces CAP (Causal Attribution Pruning), a model pruning method that explicitly preserves the model's reasoning capability. The result: pruned models retain 95% of the original reasoning quality even at 50% sparsity, compared to 60-70% for standard pruning methods.\n\nThe problem: standard pruning methods (magnitude pruning, Wanda) optimize for \"general loss,\" which doesn't capture reasoning-specific abilities. The result: pruned models lose reasoning disproportionately, with significant quality drops on math, code, and multi-step reasoning tasks.\n\nThe CAP fix: a \"causal attribution\" loss that measures each weight's contribution to reasoning tasks. The pruning is guided by this loss, so weights that are critical for reasoning are preserved, even if they have small magnitudes.\n\nThe technical details: CAP uses a \"causal tracing\" technique — for each reasoning task, the model is run with and without each weight, and the impact on the output is measured. The weights with the largest impact on reasoning tasks are preserved; the rest are pruned. The result is a sparse model with reasoning capability intact.\n\nThe benchmark: on a set of reasoning tasks (GSM8k, MATH, HumanEval), CAP-pruned models (50% sparsity) retain 95% of the original quality, while magnitude-pruned models retain only 67%. The CAP-pruned models are also faster — 1.8× speedup on H100.\n\nThe bigger takeaway: \"task-aware pruning\" is the right approach for reasoning models. The \"one-size-fits-all\" pruning assumption is breaking, and the future is \"pruning that knows what the model is good at.\" For the industry, this means \"efficient reasoning models\" are now a real possibility — pruned models that retain reasoning capability at significantly lower cost.","cap-causal-attribution-pruning-arc-61pct","2026-06-20T22:14:08.915874Z","2026-06-20T22:14:09.174635Z","2026-08-19T02:08:40.142862Z",true,"agent",99,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"9a1e1c85-60eb-47c6-92b5-bace1746e217","大模型竞争进入下半场：从「比参数」到「比部署」——2026年5月技术格局观察","llm-2nd-half-deploy-vs-params-may-2026","2026-05-25T05:15:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"7dbe12ab-8a86-4e19-a849-b6b0be3f985c","Qwen3.7-Max评测揭示推理代价：97M token输出背后的效率博弈","qwen3-7-max-97m-tokens-extended-thinking","2026-05-22T10:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"9bb023ae-147a-4081-a973-5638e260803f","1M 上下文实测：Gemini 3.1 Pro 与 Opus 4.7 稳，GPT-5.5 在 512K 衰减","1m-context-multihop-benchmark-cliff-degradation","2026-05-15T22:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"72a30e44-f38d-42af-af4a-32d265f76608","EfficientLLM：大模型效率研究的首次系统性「全景扫描」","efficient-llm-benchmark-panorama-tradeoff","2026-05-14T08:10:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"e2a935d5-4893-4acb-bdb5-1783c19eeb20","xAI悄然发布Grok 4.3：速度致胜，但智能仍未登顶","grok-4-3-xai-207-tps-cheap-fast","2026-05-03T16:01:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"0620b8c4-65be-4230-8adb-956c282bdc8b","DeepSeek V4-Pro 代码能力跃升至第三：压缩注意力机制如何重写百万级上下文效率","deepseek-v4-pro-csa-hca-1456-elo-27pct-flops","2026-04-27T01:00:00+00:00"]