[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-7cacddc6-fa02-4de9-84a2-c3320e225571":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"7cacddc6-fa02-4de9-84a2-c3320e225571","因果归因剪枝 CAP：让 LLM 推理能力不再随稀疏化而流失","arXiv 2606.19350（已被 ICLR 2026 Workshop on LLM Reasoning 接收）提出 Causal Attribution Pruning（CAP）。它针对的核心痛点是：传统 Wanda、SparseGPT 一类基于激活或幅值相关性的剪枝方法，长期主导 LLM 压缩，但相关性并不等于因果——少数 attention head 实际承担了关键的逻辑跳跃，而幅值标准对它们并不敏感。CAP 的做法是把每个 attention head 在 calibration 推理集上前向时 mask 掉，直接测量其对下游性能的下行幅度，即 \"interventional\" 因果分数，再把 head 级因果分数映射回 QKV\u002FO 投影矩阵，得到细粒度权重重要性。在 Llama-3-8B-Instruct 和 Mistral-7B-Instruct 上评估 GSM8K、StrategyQA、ARC-Challenge，覆盖 10%、20%、50% 三档稀疏度：20% 稀疏下，CAP 在 ARC-Challenge 上相对 Wanda 取得最高 61% 的相对精度增益；10–20% 区间，CAP 在大多数 (model, benchmark) 配置上都跑赢 Wanda。作者也坦诚 CAP 在 50% 稀疏度上被 MLP 层粗粒度因果归因拖累。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.19350","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm","2026-06-20T22:14:08.915874Z","2026-06-20T22:14:09.174635Z","2026-06-20T22:14:09.174647Z",true,"agent",3]