[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-pace-four-bit-recurrent-state-quantization":3,"topics-all":38,"news-related-4d36e54d-ca37-4d34-b8d9-74ebc25762cf":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"4d36e54d-ca37-4d34-b8d9-74ebc25762cf","PACE:递归状态量化损失主因是舍入频率","混合线性注意力用固定递归状态替掉 KV 缓存,状态越小并发越高,但每步舍入让误差累积。PACE 提出 post-hoc 4-bit 量化器:每 8 个 token 舍入一次、沿 key 轴共享 scale、对最敏感的 head 留 8 bit,四个 Qwen 混合模型上同等常驻位宽下分数高于 Q-Mamba DSQ。","线性注意力模型这几年最大的卖点，是把 Transformer 的 key-value 缓存换成一个**固定大小的递归状态**：上下文再长，状态也不膨胀，但每条并发序列都要占一份它。状态越小，同一张卡上的并发就越高——这是长上下文推理共同的底层账本。\n\n麻烦在于这个状态要被反复读、反复写。每生成一个 token 就降精度舍入一次，误差会像复利一样往里堆。现有状态量化方案的结论很悲观：4 bit 会掉点，除非加宽或重训。论文 [PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.29545)（Haller、Laitenberger、Akbik，10 月 21 日挂 arXiv）想证明的是：不用重训，4 bit 也能站住。\n\n## 第一个杠杆：不是每步都舍入\n\nPACE 是 post-hoc 量化器。借鉴 prefill 分块 kernel——prefill 本来就每读完一个 chunk 才处理一次状态——它把解码时的舍入周期拉长：每 8 个 token 舍入一次，中间 token 暂存在一个小的 decode buffer，flush 时推进去再重新量化。\n\nQwen3.5-9B 上，全精度状态的 GSM8K\u002FIFEval\u002FRULER-32k 平均分是 91.0；压到 4 bit、常驻 4.60 bit\u002Fvalue、**每 token 都舍入**时掉到 78.1；每 8 个 token 一次（常驻 5.28 bit）回到 82.9；每 32 个 token 一次是 85.6。Qwen3.5-4B 是同一条曲线：88.7、76.7、84.1、86.0。\n\n也就是说，INT4 的主要损失不是位宽不够，而是\"每步都舍入\"太频繁。\n\n## 第二个杠杆：大值是按列长的\n\ngated-delta 状态里的大数值**成列聚集**，因为每个 token 写入的 delta value 中总有少数通道会持续偏大。作者用对照实验印证——这些通道在每个 token 上被置换后，大值就不再构成列。既然按列聚集，量化 scale 就该在同一列内共享，即沿 key 轴而非 value 轴。\n\n同为 5.28 bit 常驻、g=32 分组，沿 value 轴的 WikiText-103 困惑度 8.158，沿 key 轴 7.719——全精度 7.713；同表 Q-Mamba 的 DSQ 是 8.320。再叠加 head 级精度分配，4B 困惑度 7.687，IFEval 81.52，高于 DAMP INT4 的 79.48。\n\n## 实测对比与代价\n\n评测覆盖 Qwen3.5-4B\u002F9B\u002F27B 与 Qwen3.6-35B-A3B。作者报告：混合 4\u002F8 位宽的 PACE-4\u002F8 在三个较大骨干上与 8-bit DAMP 相当，常驻内存约为其三分之二；全 4-bit 的 PACE-4 在同等常驻位宽下每个骨干都高于 Q-Mamba DSQ。速度上单张 H100 PCIe，batch 16 起周期式 INT4 解码的 TPOT 已低于逐 token 的 bf16。\n\n## 所以呢\n\n量化损失的主要来源是**舍入频率**，不是位宽——这个结论对所有递归状态类架构都成立，包括 gated linear attention、Mamba-2、DeltaNet 的分块形式。对做长上下文服务的人，这是不改模型、不重训、只改解码策略的吞吐路径。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.29545","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"0efa0205-51c3-4211-939c-3a8ed6f9d9f1","en","PACE: Four-Bit Recurrent State Quantization","PACE: a post-hoc four-bit quantizer for linear-attention states. Round every 8 tokens, share scales along the key axis, keep sensitive heads at 8-bit.","Linear-attention models have been sold on one promise: replace the Transformer's key-value cache with a **fixed-size recurrent state**. Context length no longer inflates memory, but every concurrently decoded sequence still holds one copy — so shrinking the state directly buys batch size on a given accelerator. The catch is that this state is read and rewritten at every step. Rounding it to low precision after each generated token lets error compound, and existing state quantizers report a grim verdict: four bits lose accuracy unless you raise the width or fine-tune. [PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.29545) (Haller, Laitenberger, Akbik; submitted October 21) argues the opposite: four bits can hold without retraining.\n\n## First lever: stop rounding every step\n\nPACE is a post-hoc quantizer. Borrowing from chunked kernels — prefill already processes the state once per chunk — it stretches that interval into decoding: round the state every eight generated tokens, park the in-between tokens in a small decode buffer, and at each flush advance the buffer into the state before requantizing.\n\nThe numbers show the effect size. On Qwen3.5-9B, the full-precision state averages 91.0 across GSM8K, IFEval, and RULER-32k. Compressed to four bits at 4.60 resident bits per value and rounded every token, that drops to 78.1. Rounding every eight tokens (5.28 resident bits) recovers 82.9; every thirty-two tokens (7.64 resident bits) reaches 85.6. Qwen3.5-4B traces the same curve: 88.7, 76.7, 84.1, 86.0.\n\nMost of the INT4 loss, in other words, comes from *how often* you round, not from the four-bit width itself.\n\n## Second lever: large entries gather into columns\n\nThe paper also reports a structural observation: in the gated-delta state, large entries **cluster into columns**, because a few channels of the delta value written at each token stay large across many tokens. The authors confirm it by permuting those channels every token — the large entries then stop forming columns. If they cluster by column, the quantization scale should be shared within a column, along the key axis rather than the value axis.\n\nThe ablation is clean. At the same 5.28 resident bits with g=32 groups, value-axis grouping gives a WikiText-103 perplexity of 8.158 while key-axis grouping gives 7.719 — against 7.713 for the full-precision state. Q-Mamba's DSQ sits at 8.320 in the same table. Adding head-level precision allocation takes the 4B perplexity to 7.687 and IFEval to 81.52, above DAMP INT4's 79.48.\n\n## Measured comparison and its costs\n\nEvaluation covers four hybrid models from 4B to 35B parameters — Qwen3.5-4B\u002F9B\u002F27B and Qwen3.6-35B-A3B — with every configuration except DAMP INT4 rounding every eight tokens. The authors report that PACE-4\u002F8, with mixed four- and eight-bit heads, is competitive with eight-bit DAMP on the three larger backbones at about two thirds of its resident memory, and that fully four-bit PACE-4 scores higher than Q-Mamba DSQ on every backbone at equal resident bits. For speed, they time TPOT on a single H100 PCIe GPU at a 512-token context across batch sizes 1 to 128: periodic INT4 decoding is cheaper than per-token bf16 decoding from batch 16 upward, and the gap widens with batch size.\n\nThe limits are stated too: head decays, column structure, and head ranking are measured only on these four Qwen hybrid models; the prefill rounding period is fixed at the kernel's 256-token chunk; and throughput is timed only for the uniform four-bit state.\n\n## So what\n\nThe value here is not \"four bits got a bit more workable.\" It is a different framing of the problem: the dominant source of quantization loss is **rounding frequency**, not the bit width itself — a conclusion that applies to every recurrent-state architecture, including the chunked forms of gated linear attention, Mamba-2, and DeltaNet. For teams running hybrid linear-attention models in long-context serving, that is a throughput path requiring no model changes, no retraining, and only a different decoding schedule.","pace-four-bit-recurrent-state-quantization","2026-10-23T01:10:00Z","2026-10-23T01:17:23.121349Z","2026-10-23T01:17:23.121356Z",true,"agent",8,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"be76fca7-bc02-4a76-a58f-17a661b82b60","SparPR:消费级显卡的KV缓存回收术","sparpr-kv-cache-recovery-consumer-gpu","2026-10-22T01:20:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"5c1d994e-4baf-41ae-a57d-33411a8981b5","KAIST KV4-RL:INT4量化KV缓存,RL训练rollout提速1.8倍","kv4-rl-int4-kv-cache-rlvr-rollout","2026-10-15T19:11:53+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00"]