[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-d-quant-entropy-coding-kv-cache":3,"topics-all":38,"news-related-3cecce90-70b9-4bb3-b9b7-93e6b0c05105":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","腾讯团队 arXiv 发 D-Quant,把 rANS 熵编码搬进 KV cache 量化:定长容器加 drift 机制,2.26bit 逼近 BF16 精度,64K 长上下文 KV 内存省约 7 倍,单卡 H20 吞吐 563.6 tokens\u002Fs,免校准数据。","KV cache 量化这几年的主线,基本是\"更聪明的预处理 + 更低的固定位宽\":旋转、归一化、换分组粒度,但编码格式本身没人动过——大家默认每个值都得占同样多的比特。腾讯团队刚放上 arXiv 的 D-Quant 换了个思路:把视频压缩里用了几十年的熵编码(rANS)搬进 attention 内核,让高频值用短码、低频值用长码,再用一个\"drift\"机制把变长输出掰回定长容器。结果是在 2.26 bit 每值的预算下逼近 BF16 精度,64K 长上下文 KV 内存省约 7 倍。\n\n## 为什么固定位宽是天花板\n\n固定宽度量化的根本限制是:b bit 只能表示 2^b 个量化水平。往 2-bit 走,水平数掉到 4 个,信息损失断崖式下跌。而 D-Quant 的观察是:KV 值经过旋转和归一化后近似正态分布——大部分值挤在中心,尾部极少。固定宽度编码却给高频符号和稀有符号同样的比特数,等于白扔了这份分布结构。熵编码天然吃这种非均匀分布:按符号概率分配码长,同样的平均存储预算下可以塞进多得多的量化水平。\n\n麻烦在工程侧。熵编码输出变长码流,而 attention 内核的高效反量化和计算依赖规则内存布局、定长寻址——变长数据在并行 kernel 里是灾难。之前 CacheGen、SplitZip 这类工作只敢把熵编码用于 KV 传输(先解压再算),没人敢直接放进计算路径。\n\n## drift:把变长掰回定长\n\nD-Quant 的解法是给每个 token 的 K 或 V 张量分配一个定长字节容器,把量化后的符号用 rANS 熵编码塞进去;塞不下时,少量符号从高成本符号\"漂移\"到邻近的低成本符号,直到码长满足约束。论文测得漂移只改动 0.75% 的 key 符号和 5.00% 的 value 符号,且没有任何符号移动超过一个重建水平——是稀疏局部修正,不是全局降精度。\n\n这个设计还顺手解开了两个死结。其一,量化水平数与位宽解耦:2.26 bit 预算下,key 用 8 个水平、value 用 6 个,固定位宽 paradigm 里这不可想象。其二,免校准:预处理后 KV 分布足够接近标准正态,概率模型直接从解析式导出,不需要标定数据集。\n\n## 数字说话\n\nQwen3-8B 上 RULER 平均分 86.50(BF16 为 87.31),Llama-3.1-8B 上 88.94(参考 89.69);128K 上下文时 D-Quant 拿到 70.92 \u002F 74.65,最强 baseline 只有 57.14 \u002F 66.15。效率侧,8K prompt 下 BF16 在 batch 64 爆显存,D-Quant 跑到 batch 128 峰值内存仅 49.4 GiB;各自最大 batch 下吞吐 563.6 对 159.5 tokens\u002Fs,提升 3.5 倍。把预算再压到 1.96 bit,Qwen3-8B 仍保 85.21 分,继续压过全部 baseline。\n\n值得注意的是这笔交易的另一面:熵解码带来额外计算,小 batch 下 D-Quant 比 BF16 慢——它换的是显存受限场景下的 serving 容量,不是单请求延迟。这个权衡在长上下文 Agent 场景(高并发、长会话)恰好是对的位置。\n\n## 所以呢\n\nKIVI、KVQuant、QuaRot 这条线一直在优化\"怎么摆 outlier\",D-Quant 换了进攻方向:编码格式本身还有多少水分可挤。对做推理服务的团队,7 倍 KV 内存压缩加 3.5 倍吞吐增益,值得立刻评估接入成本;对研究者,熵编码进入 attention kernel 这一步打通后,速率-失真优化这套视频压缩的成熟工具箱,接下来都可以往 KV cache 上搬。论文:arxiv.org\u002Fabs\u002F2609.19880。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19880","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"9341b54d-c1d2-4462-a8c3-f3e9fde2e10c","en","D-Quant: Entropy Coding Meets KV Cache at 2.26 Bits","Tencent's D-Quant on arXiv: rANS entropy coding for KV cache, near-BF16 at 2.26 bits, ~7x less KV memory at 64K, 563.6 tok\u002Fs on one H20, no calibration data.","KV cache quantization has spent years on smarter preprocessing and lower fixed bit widths — rotation, normalization, finer grouping — while the coding format itself went untouched: every value gets the same number of bits by default. D-Quant, just posted on arXiv by a Tencent team, takes a different route: it brings rANS entropy coding, a staple of video compression for decades, into the attention kernel, letting frequent values take short codewords and rare ones long codewords, with a \"drift\" mechanism bending variable-length output back into fixed-size containers. The result: near-BF16 quality at 2.26 bits per value, roughly 7x less KV memory at 64K long context, and 563.6 tokens\u002Fs throughput on a single H20.\n\n## Why Fixed Width Is a Ceiling\n\nThe fundamental limit of fixed-width quantization is that b bits can only represent 2^b quantization levels. Push toward 2 bits and the level count collapses to 4, with cliff-edge information loss. D-Quant's observation: after rotation and normalization, KV values approximately follow a normal distribution — most values cluster near the center, with only a small fraction in the tails. Fixed-width coding assigns the same bits to frequent and rare symbols alike, throwing away that distributional structure. Entropy coding naturally exploits non-uniformity: codeword lengths follow symbol probabilities, so far more quantization levels fit under the same average storage budget.\n\nThe trouble is engineering. Entropy coding produces variable-length streams, while efficient dequantization and computation in attention kernels rely on regular memory layouts and fixed-stride access — variable-length data is a disaster in parallel kernels. Prior work like CacheGen and SplitZip only dared use entropy coding for KV transfer (decompress first, compute later); nobody put it directly on the compute path.\n\n## Drift: Bending Variable Back to Fixed\n\nD-Quant assigns each token's K or V tensor a fixed-size byte container and entropy-codes the quantized symbols into it with rANS. When a stream exceeds the budget, a small fraction of symbols drift from high-cost symbols to nearby lower-cost ones until the length constraint holds. The paper measures that drift changes only 0.75% of key symbols and 5.00% of value symbols, and no changed symbol moves by more than one reconstruction level — sparse local corrections, not a global precision cut.\n\nThe design also unties two old knots. First, quantization level count is decoupled from bit width: at a 2.26-bit budget, keys use 8 levels and values 6 — unthinkable under the fixed-width paradigm. Second, no calibration: preprocessed KV distributions are close enough to a standard normal that the probability model derives analytically, no calibration dataset needed.\n\n## The Numbers\n\nOn Qwen3-8B, D-Quant scores 86.50 RULER average (BF16 reference: 87.31); on Llama-3.1-8B, 88.94 (reference: 89.69). At 128K context, D-Quant reaches 70.92 \u002F 74.65 while the strongest baselines manage only 57.14 \u002F 66.15. On efficiency: with an 8K prompt, BF16 runs out of memory at batch 64 while D-Quant scales to batch 128 with just 49.4 GiB peak memory; at respective maximum batches, throughput is 563.6 versus 159.5 tokens\u002Fs, a 3.5x gain. Push the budget down to 1.96 bits and Qwen3-8B still holds 85.21, again beating every baseline.\n\nWorth noting is the other side of the trade: entropy decoding adds computation, so D-Quant is slower than BF16 at small batch sizes — what it buys is serving capacity under memory-constrained scenarios, not single-request latency. That trade lands exactly right for long-context agent workloads with high concurrency and long sessions.\n\n## So What\n\nThe KIVI \u002F KVQuant \u002F QuaRot line kept optimizing how to handle outliers; D-Quant shifted the axis of attack: how much water is still left in the coding format itself. For inference-serving teams, 7x KV memory compression plus 3.5x throughput is worth an immediate integration-cost assessment. For researchers, once entropy coding is inside the attention kernel, the mature rate-distortion toolbox from video compression can now be carried over to KV cache. Paper: arxiv.org\u002Fabs\u002F2609.19880.","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42Z","2026-09-20T17:10:52.082744Z","2026-09-20T17:10:52.082758Z",true,"agent",1,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"77015cf0-fb2c-4176-ab77-f428d8bd2d30","UltraQuant 把 KV Cache 压到 4-bit：Agentic 长上下文推理首次跑出 3.47× TTFT 加速","ultraquant-amd-4bit-kv-cache-3-47x","2026-06-22T18:15:00+00:00"]