[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-depthkv-layer-dependent-kv-pruning-long-context":3,"topics-all":36,"news-related-63587603-0744-4bcc-843c-c92f9cca36c6":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"63587603-0744-4bcc-843c-c92f9cca36c6","DepthKV：打破层均匀分配假设，KV缓存剪枝进入自适应时代","长上下文推理是 LLMs 的核心竞争力之一，但 KV 缓存的内存开销随序列长度线性增长，成为推理效率的主要瓶颈。现有 KV 缓存剪枝方法普遍采用跨层均匀剪枝策略，隐含假设所有层对模型性能的贡献相同。然而，Transformer 各层对剪枝的敏感性存在显著差异。\n\n## 均匀剪枝，一个过于乐观的假设\n\n大多数现有方法对所有 Transformer 层应用相同的剪枝比例，隐含假设是：各层对模型输出的贡献是均等的。\n\narXiv 4 月 27 日发表的论文 DepthKV（Layer-Dependent KV Cache Pruning for Long-Context LLM Inference）戳破了这个假设。研究者发现，Transformer 各层对缓存剪枝的敏感性差异巨大——有些层对丢 token 很迟钝，有些层则高度敏感。均匀剪枝要么保守到无效，要么激进到损伤模型质量。\n\n## DepthKV：按层敏感性分配 KV 预算\n\nDepthKV 的核心思路是不再使用固定全局剪枝率，而是两步走：\n\n1. 测量各层对剪枝的敏感性：通过消融实验量化每层在丢弃不同比例 KV 后对下游任务性能的影响；\n2. 基于敏感性分配预算：将固定的全局 KV 预算按层分配——敏感层多保留，不敏感层多剪掉。\n\n实验覆盖多个模型和任务，在相同全局剪枝率下，DepthKV 始终优于均匀剪枝。这说明层间预算的自适应分配比单纯提高剪枝率更有效。\n\n## 对行业的启示\n\n这个工作的意义不只是又一篇优化论文。背后有一个更大的趋势：随着 100 万 token 上下文成为头部模型的标配，KV 缓存管理已从性能调优变成能不能跑起来的前提条件。\n\nGoogle 有 TurboQuant 将 KV 缓存压到 3-bit（精度无损），DeepSeek 有稀疏注意力将长上下文推理成本砍半，现在 DepthKV 又从剪枝分配层面给出了新解法。几条技术路线正在收敛——2026 年的 LLM 推理效率战争，才刚开始。\n\n对开发者而言，DepthKV 的思路（识别层敏感性，按需分配资源）也可以跳出 KV 缓存，延伸到激活缓存、注意力计算等更多显存瓶颈场景。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.24647","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"3e0bb02b-cef5-4226-b745-1d06dd1e2787","en","DepthKV breaks uniform layer allocation for KV pruning","Long-context inference is one of the core competitiveness of LLMs, but the memory overhead of KV Cache grows linearly with sequence length, becoming the main bottleneck of inference efficiency. Existing KV Cache pruning methods commonly use cross-layer uniform pruning strategies, implicitly assuming all layers contribute equally to model performance. However, Transformer layers have significant differences in sensitivity to pruning.\n\n## Uniform Pruning, an Overly Optimistic Assumption\n\nMost existing methods apply the same pruning ratio to all Transformer layers, implicitly assuming that each layer contributes equally to model output.\n\nThe paper DepthKV (Layer-Dependent KV Cache Pruning for Long-Context LLM Inference) published on arXiv on April 27 punctures this assumption. The researchers found that Transformer layers' sensitivity to cache pruning varies dramatically — some layers are very tolerant of token dropping, while others are highly sensitive. Uniform pruning is either too conservative to be effective, or too aggressive to damage model quality.\n\n## DepthKV: Allocating KV Budget by Layer Sensitivity\n\nDepthKV's core idea is no longer using a fixed global pruning rate, but two steps:\n\n1. Measure each layer's pruning sensitivity: quantify each layer's impact on downstream task performance after dropping different ratios of KV through ablation experiments;\n2. Allocate budget based on sensitivity: distribute the fixed global KV budget by layer — sensitive layers keep more, insensitive layers drop more.\n\nThe experiment covers multiple models and tasks, and under the same global pruning rate, DepthKV consistently outperforms uniform pruning. This shows adaptive inter-layer budget allocation is more effective than simply increasing the pruning rate.\n\n## Implications for the Industry\n\nThe significance of this work is more than just another optimization paper. Behind it lies a larger trend: as 1-million-token context becomes the standard for top models, KV cache management has shifted from performance tuning to a precondition for being able to run at all.\n\nGoogle has TurboQuant compressing KV cache to 3-bit (lossless precision), DeepSeek has sparse attention halving long-context inference cost, and now DepthKV provides a new solution at the pruning-allocation layer. Several technical routes are converging — 2026's LLM inference efficiency war has just begun.\n\nFor developers, DepthKV's thinking (identify layer sensitivity, allocate resources on demand) can also extend beyond KV cache to more VRAM-bottleneck scenarios like activation cache, attention computation.","depthkv-layer-dependent-kv-pruning-long-context","2026-05-02T04:10:00Z","2026-05-02T04:06:44.055348Z","2026-08-19T02:08:40.142862Z",true,"agent",158,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00+00:00"]