[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-stepquant-delta-rule-state-quantization":3,"topics-all":38,"news-related-33f9c56d-1bcd-403d-85e2-273fb3d498fb":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"33f9c56d-1bcd-403d-85e2-273fb3d498fb","循环状态量化破局:STEPQuant 6-bit 贴平 FP32,显存省 68.7%","浙大团队提出 STEPQuant,对 Delta-rule 循环状态做时空双维量化:按误差寿命分配比特,key 行与 value 列分别拟合 scale。Qwen3.8-27B 上 6-bit 贴平 FP32,4-bit 胜均匀 INT8;集成 SGLang 后状态压缩超 5 倍,总显存最多降 68.7%。","线性注意力把随长度增长的 KV cache 换成了固定大小的循环状态,但\"固定\"只对单个请求成立:并发一上来,每个请求都要留一份状态,显存照样线性堆。更麻烦的是,把状态直接压到低比特,量化误差会被一次次状态更新持续放大——这是 Delta-rule 混合架构落地 serving 时留下的一个工程死角。浙江大学的团队在论文 arXiv 2609.38169 里提出 STEPQuant,给出了系统解法(https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.38169)。\n\n## 误差分两维:活多久,影响谁\n\n论文先回答\"为什么不能均匀量化\"。时间维上,长寿命记忆单元的误差会跨多个解码步存留;空间维上,不同 key 行对输出的影响差异很大,而状态幅值在行、列两个方向都剧烈波动。均匀比特分配在这两维上都押错宝:既给不重要的单元浪费精度,又让高风险单元带着误差滚雪球。\n\n## 按误差花比特,按影响定 scale\n\nSTEPQuant 是一个后训练量化框架:按误差幅度与记忆寿命分配比特,少量高危单元直接保留 FP16;key 行与 value 列分别拟合 scale,优先保护对读出误差影响大的行。校准只需一次,配合 packed-state kernel 与异步写回保持推理效率。\n\n## 数字:6-bit 贴平 FP32\n\n七项长生成任务、BF16 权重加量化状态的设定下,Qwen3.8-27B 的 FP32 平均准确率 80.60%,均匀 INT8 掉到 71.86%,均匀 INT6 直接崩至 45.04%;STEPQuant@6 拿到 80.59%,STEPQuant@4 仍有 80.51%。Kimi-Linear-48B-A3B-Instruct 上结论同向:FP32 61.52%,STEPQuant@6 61.47%,STEPQuant@4 58.52%,均高于均匀 INT8 的 56.02%。工程侧,集成 SGLang(测试栈 SGLang 0.5.12 + PyTorch 2.11.0 + Triton 3.7.1,任务评测在 NVIDIA A800 上完成)后,6-bit 配置把循环状态压缩 5.03×\u002F5.08×(Qwen\u002FKimi),总 serving 显存最多降 68.7%。\n\n## 所以呢\n\n九月上旬同赛道刚有工作论证了 Gated DeltaNet 全压 4-bit 可行,STEPQuant 则把\"怎么压好\"推进到可复现工程:代码与复现文档已开源(https:\u002F\u002Fgithub.com\u002FDreamer-Toby\u002FSTEPQuant)。值得注意的是论文的诚实披露——4\u002F6-bit 是名义预算,FP16 pivot 与 scale 本身要占额外存储;显存口径用了 W4\u002FAWQ 权重加每请求五个前缀状态槽。对做混合架构推理优化的团队,这篇的时空双维分析框架比跑分本身更值得搬走:量化不再是对着张量一刀切,而是先问误差会活多久、会影响谁。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.38169","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e9059fe3-e93d-4c56-af00-47a218b6a845","en","STEPQuant: 6-bit Recurrent States Match FP32, Memory Down 68.7%","ZJU's STEPQuant quantizes Delta-rule recurrent states; 6-bit matches FP32, 4-bit beats INT8 on Qwen3.8-27B, SGLang serving memory down 68.7%.","Linear attention swaps the ever-growing KV cache for a fixed-size recurrent state, but \"fixed\" only holds per request: under concurrent serving, every request keeps its own state and memory still piles up. Worse, directly quantizing those states to low precision lets errors compound through successive state updates. A Zhejiang University team tackles this engineering dead spot with STEPQuant (https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.38169).\n\n## Errors have two dimensions: how long they live, and whom they hit\n\nThe paper first answers why uniform quantization fails. Temporally, errors in long-lived memory units persist across many decoding steps. Spatially, different key rows affect outputs very differently, while state magnitudes swing along both rows and columns. Uniform bit allocation wastes precision on unimportant units while letting high-risk ones accumulate error like a snowball.\n\n## Spend bits by error, set scales by impact\n\nSTEPQuant is a post-training quantization framework: it allocates precision by error magnitude and memory lifetime, keeps a few high-risk units in FP16, and fits key-row and value-column scales separately, prioritizing rows that matter most to readout error. Calibration happens once; packed-state kernels and asynchronous writeback keep inference efficient.\n\n## The numbers: 6-bit matches FP32\n\nAcross seven long-generation tasks with BF16 weights and quantized states, Qwen3.8-27B scores 80.60% average accuracy at FP32, drops to 71.86% under uniform INT8, and collapses to 45.04% under uniform INT6; STEPQuant@6 hits 80.59% and STEPQuant@4 still reaches 80.51%. Kimi-Linear-48B-A3B-Instruct shows the same direction: FP32 61.52%, STEPQuant@6 61.47%, STEPQuant@4 58.52%, all above uniform INT8's 56.02%. On the engineering side, integrated into SGLang (tested with SGLang 0.5.12, PyTorch 2.11.0, and Triton 3.7.1; task evaluations ran on NVIDIA A800 GPUs), the 6-bit configuration compresses recurrent states by 5.03x\u002F5.08x (Qwen\u002FKimi) and cuts total serving memory by up to 68.7%.\n\n## So what\n\nEarlier this September, work on Gated DeltaNet showed full 4-bit compression of these hybrid models is feasible; STEPQuant pushes the \"how to do it well\" part into reproducible engineering, with code and reproduction docs open-sourced (https:\u002F\u002Fgithub.com\u002FDreamer-Toby\u002FSTEPQuant). The disclosures are notably honest: the 4\u002F6-bit budgets are nominal, since FP16 pivots and scales add storage, and the memory accounting uses W4\u002FAWQ weights with five prefix-state slots per request. For teams optimizing hybrid-architecture inference, the spatio-temporal analysis framework matters more than the scores themselves: quantization is no longer one-size-fits-all on a tensor, but starts by asking how long an error lives and whom it affects.","stepquant-delta-rule-state-quantization","2026-10-08T21:07:39Z","2026-10-08T21:07:56.698818Z","2026-10-08T21:07:56.698833Z",true,"agent",124,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"3e28cfaf-74aa-4521-8eba-37332fe93901","7块ESP32跑1.58-bit Qwen:功耗1.53瓦","esp32s3-bitnet-llm-cluster","2026-09-30T13:11:01+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"d056f67b-7e0d-4e44-8d39-e31ea50deeae","Bonsai 2 27B 三元压缩:Qwen3.8 压到 5.9 GB,benchmark 留存 98.2%","bonsai-2-27b-ternary-qwen3-8-compression","2026-09-17T15:47:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"2266cea6-06f1-4932-8905-1bc3f2e5a8c0","Meta FAIR 字节蒸馏研究:End-Of-Token 渐近反超 token 蒸馏 4%,数据只需 1\u002F6","meta-fair-byte-distillation-token-ceiling-2026-09","2026-09-15T02:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"49c90242-7793-46c7-961a-8a39e608e23d","竖线 token 才是量化命门：HyQuant 混合精度让 4-bit 注意力几乎无损","hyquant-hybrid-precision-attention-quantization","2026-09-13T17:06:39+00:00"]