[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-latent-compaction-test-time-kv":3,"topics-all":38,"news-related-a046b158-396f-4db9-a8d3-22d8c193a166":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"a046b158-396f-4db9-a8d3-22d8c193a166","清华把KV缓存当神经网络训练:32倍压缩准确率96.7%","清华THUNLP团队提出Latent Compaction，把长文档的KV缓存当作可训练的小神经网络：模型权重全程冻结，只用文档相关通用提示词做测试时训练，不接触下游问答。RULER基准上以32倍压缩取得96.7%检索准确率，同预算KVZip+仅40.0%；方法在Qwen3与MiniCPM5上均有效，代码MIT开源。","长文档塞进大模型，最先撑爆的不是参数而是 KV 缓存——每多读一个 token，缓存就多一行键值对，内存占用和注意力计算成本跟着膨胀。过去几年的主流答案是「压缩」：丢掉不重要的条目，或者把缓存量化到低精度。清华 THUNLP 团队 10 月 14 日提交到 arXiv 的论文 Latent Compaction 给出了另一条路：不挑不扔，直接把 KV 缓存当作一个可训练的小神经网络，用测试时训练为每篇文档现学出一个紧凑缓存。\n\n## 把缓存当神经网络，两阶段训练\n\n方法分两步。初始化用三个信号挑出种子条目：重建重要性、教师注意力、键空间多样性。第一阶段让压缩缓存的注意力输出和 log-normalizer 逼近完整缓存的对应值，重建选中的条目全程固定；第二阶段通过从完整缓存模型蒸馏，端到端联合优化整个缓存。GitHub README 给出的默认训练计划是两个阶段各 1000 次更新。\n\n两个约束贯穿始终：模型权重全程冻结，动的只有缓存本身；训练只用关于文档的通用提示词，完全不接触下游问题和答案。压缩后的缓存因此是「文档专属资产」——一次训练，之后反复用来回答大量没见过的问题。\n\n## 32 倍压缩下的 96.7% 对 40.0%\n\n论文在 RULER、LongBench 和 Phonebook Lookup 三个基准上与同预算压缩基线对比。论文报告的最亮眼数字在 RULER：32 倍压缩下取得 96.7% 的检索准确率，同预算的 KVZip+ 只有 40.0%。复现包相当完整——KVZip+、Attention Matching、SnapKV、PyramidKV 加一个文本摘要基线全部跑在同一个 runner 上，复现命令直接写在 README 里，Phonebook 实验覆盖 4096 到 32768 的上下文长度、2 到 256 倍的压缩档位。方法还跨模型规模与家族迁移，在 Qwen3 和 MiniCPM5 上都有检索增益，快速上手示例用的就是 Qwen3-1.7B。\n\n## 代价与赛道位置\n\n免费午餐不存在。这个方法要为每篇文档付出一次测试时训练成本（论文默认两个阶段各 1000 步），换来之后所有问题共享一个 1\u002F32 大小的缓存。单次问答场景它是亏的；「一份长文档反复查询」的场景——客服知识库、合同库、代码库——摊销逻辑立刻成立。\n\n放进赛道坐标：量化派押注表示精度，驱逐派（SnapKV 一系）押注选择启发式，Latent Compaction 押注的是缓存本身有可学习结构。测试时训练被热炒后，多数工作拿它增强推理，清华团队把它用在压缩上，给这条技术线找到了一个工程出口。代码以 MIT 协议开源，论文 19 页 21 图见 arXiv:2610.20582（https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.20582），仓库在 github.com\u002FTHUNLP\u002Flatent-compaction。\n\n一句话收束：当缓存从模型的附属品变成可优化的对象，长上下文的成本公式就被改写了一项——剩下的问题只是，每篇文档值不值 2000 步训练。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.20582","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e4306716-ee8b-4800-abfb-c14af910f40c","en","Tsinghua's Latent Compaction: 96.7% Accuracy at 32x KV Squeeze","Tsinghua trains each document's KV cache at test time, weights frozen: 96.7% RULER accuracy at 32x compression vs 40.0% for KVZip+, code open-sourced.","Feed a long document to a large model and the first thing to burst is not the parameters but the KV cache: every extra token adds another row of key-value entries, inflating memory and attention cost. The mainstream answers of recent years have been compression strategies — evict unimportant entries, or quantize the cache to lower precision. A paper submitted to arXiv on October 14 by Tsinghua's THUNLP team, Latent Compaction, takes a different route: stop picking and discarding, and instead treat the KV cache as a small trainable neural network, using test-time training to learn a compact cache for each document.\n\n## Treat the Cache as a Neural Network, Train in Two Stages\n\nThe method runs in two steps. Initialization selects seed entries using three signals: reconstruction importance, teacher attention, and key-space diversity. Stage one fits the compact cache's attention outputs and log-normalizers to those of the full cache, with reconstruction-selected entries held fixed. Stage two jointly optimizes the entire cache through end-to-end distillation from the full-cache model. The default schedule in the GitHub README is 1,000 updates per stage.\n\nTwo constraints hold throughout: model weights stay frozen — only the cache moves — and training uses only generic prompts about the document, with no access to downstream questions or answers. The compacted cache is therefore a document-specific asset: train once, then reuse it to answer many unseen questions.\n\n## 96.7% versus 40.0% at 32x Compression\n\nThe paper evaluates on RULER, LongBench, and Phonebook Lookup against budget-matched compression baselines. Its headline number is on RULER: 96.7% retrieval accuracy at 32x compression, while budget-matched KVZip+ reaches 40.0%. The reproduction package is thorough — KVZip+, Attention Matching, SnapKV, PyramidKV, and a text-summary baseline all run on a single runner, with reproduction commands in the README. Phonebook experiments span context lengths from 4,096 to 32,768 and compression ratios from 2x to 256x. The method also transfers across model scales and families, with retrieval gains on both Qwen3 and MiniCPM5; the quick-start example uses Qwen3-1.7B.\n\n## Cost and Where It Sits in the Race\n\nThere is no free lunch. Each document pays a one-time test-time training cost (1,000 updates per stage by default) in exchange for every future question sharing a cache 1\u002F32 the size. For single-shot question answering it loses money; for workloads that query the same long document repeatedly — support knowledge bases, contract repositories, codebases — the amortization logic holds immediately.\n\nPlaced on the race map: the quantization camp bets on representational precision, eviction methods (the SnapKV line) bet on selection heuristics, and Latent Compaction bets that the cache itself has learnable structure. While most test-time-training work aims at boosting reasoning, the Tsinghua team points the technique at compression, giving the line a concrete engineering outlet. The code is MIT-licensed, and the 19-page paper is at arXiv:2610.20582 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.20582), with the repository at github.com\u002FTHUNLP\u002Flatent-compaction.\n\nThe one-line takeaway: once the cache stops being an accessory and becomes an optimizable object, one term in the long-context cost formula is rewritten — the only question left is whether each document is worth 2,000 training steps.","latent-compaction-test-time-kv","2026-10-15T17:08:40Z","2026-10-15T17:08:43.061601Z","2026-10-15T17:08:43.061610Z",true,"agent",174,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"dab2e487-cfba-4d75-a93f-eff133476348","剪枝校准对不上解码分布?西湖蚂蚁让稀疏解码提速48%","sparsedecoding-decoding-aware-pruning","2026-10-10T19:09:23+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"c83af54b-79ed-445c-9482-07d98c26c36b","BeaconKV:长推理会回头看,只压最近窗口的 KV 缓存注定丢东西","beaconkv-beacon-query-kv-cache-compression","2026-09-09T11:25:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"bce9fc16-d31a-49be-b17b-f144619a58e2","LatentPress:上下文压成软令牌直读，7.7 倍压缩反超原文，训练仅动 0.1% 参数","latentpress-soft-token-context-compression","2026-09-05T19:06:09+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"d31bc388-b6c7-41a5-a6e9-6f00657c7616","加GPU还是压KV缓存？arXiv论文：压缩省钱1.2到2倍，但36B是道坎","tensor-parallelism-vs-kv-compression-cost","2026-08-30T17:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","reskv-residual-kv-cache-compression","2026-08-03T10:43:23+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00"]