[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-kv4-rl-int4-kv-cache-rlvr-rollout":3,"topics-all":38,"news-related-5c1d994e-4baf-41ae-a57d-33411a8981b5":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"5c1d994e-4baf-41ae-a57d-33411a8981b5","KAIST KV4-RL:INT4量化KV缓存,RL训练rollout提速1.8倍","RL 训练的算力大头在 rollout。KAIST 发现异步并发把上下文压进更少解码步,KV 缓存成新瓶颈,直接 INT4 量化又伤稳定性。KV4-RL 用 INT4 KV 缓存加关键位置 INT8 保护:Qwen3-8B GRPO 训练提速 1.80 倍,精度差不到 0.5 个百分点。","RL 大模型训练的成本大头,不在梯度更新,而在 rollout:模型要自己生成大量长推理轨迹,再拿可验证奖励去打分。生成量一上来,显存和带宽就被 KV 缓存吃掉——而异步 rollout 架构把这个问题放大了。\n\n## 异步 rollout 的隐藏瓶颈:KV 集中\n\nKAIST 团队(作者 Sihwan Park、Sung-Yub Kim、Doohyuk Jang、Eunho Yang)在 10 月 10 日提交到 arXiv 的论文里给这个现象起了名字:**KV concentration(KV 集中)**。异步 rollout 靠持续并发填满 GPU、减少长尾生成时的空转;但并发一高,上下文处理被压进更少的解码步,每一步要搬运的 KV 工作量随之膨胀。团队在匹配负载的对照实验里还发现:这种场景下,KV 缓存量化带来的加速远大于只量化权重或权重+激活的方案——瓶颈确实在 KV,不在权重。\n\n最激进的 INT4 KV 量化当然最快,但论文实测它会伤学习稳定性,即便配上 token 级截断重要性采样(TIS)也救不回来:采样分布被压歪,训练信号就跟着歪。\n\n## INT4 打底,给关键位置上 INT8 保险\n\nKV4-RL 的解法是分层保护:主体上下文用免校准的 INT4 KV 量化存储,论文正文描述为分页 INT4 格式;只给两类关键位置保留 INT8 侧缓存——attention sink(注意力锚点)和近期上下文,并用更高精度的概率值算术处理这些位置。受保护的位置数有上界,既把采样分布的失真压回去,又保住了 INT4 的大部分加速。\n\n## 1.8 倍加速,精度差不到 0.5 个百分点\n\n数字全部来自 Qwen3-8B-Base 的 400 轮 GRPO 训练:rollout 累计提速 **1.80 倍**,整轮迭代(含训练)提速 **1.27 倍**,最后 50 轮的 rollout 提速冲到 **1.97 倍**。精度侧,在同一个 BF16 评测器下,Qwen3-4B-Base 和 Qwen3-8B-Base 两个规格的六基准平均精度,与全 BF16 rollout 训练的差距都在 **0.5 个百分点**以内。论文 32 页、15 张图、6 张表,CC-BY 4.0 许可,HF papers 页面已挂出一个配套 Space,社区热度还在爬升。\n\n## 推理侧卷了很久,训练侧刚开场\n\n过去一个月这个赛道并不冷清——把 KV 缓存当神经网络训练、逐层调压缩率、把 KV 挪出显存、熵编码压到 2 bit 级,本站都陆续覆盖过,但清一色是推理侧(serving)的故事。KV4-RL 换了条轴:RL 训练的 rollout 阶段本身就是\"生成即服务\",异步调度让 KV 的成本结构完全不同,而这类训练侧 KV 工程此前几乎没有系统研究,论文的匹配负载对照(KV 量化 vs 权重量化 vs 权重+激活量化)填的正是这个空档。\n\n对正在跑 RLVR 的团队,这是笔能直接算的账:rollout 占整轮时间的比例越高,1.8 倍的 rollout 加速折算到端到端就越可观;而\"几乎不丢精度\"意味着不用动训练配方就能直接上。当 RL 后训练成为大模型的标配流水线,rollout 侧的显存与带宽工程,大概率是下一个被卷起来的角落。\n\n参考:arXiv:2610.14448(https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.14448)· HF papers(https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2610.14448)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.14448","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"0f0b9350-a017-412e-a565-556292ea55cc","en","INT4 KV cache speeds RL rollouts 1.8x at under 0.5pp cost","KAIST's KV4-RL caches RL rollouts in INT4 KV with INT8-protected sinks: 1.80x speedup on Qwen3-8B GRPO, within 0.5pp of BF16 accuracy.","The dominant cost of RL training for large models is not the gradient update but the rollout: the model has to generate large volumes of long reasoning trajectories, then score them with verifiable rewards. As generation scales, KV cache eats the memory and bandwidth — and asynchronous rollout architectures amplify the problem.\n\n## The hidden bottleneck of async rollout: KV concentration\n\nA KAIST team (authors Sihwan Park, Sung-Yub Kim, Doohyuk Jang, Eunho Yang) named this phenomenon in a paper submitted to arXiv on October 10: **KV concentration**. Asynchronous rollout keeps GPUs busy through sustained concurrency, cutting idle time during long-tail generation. But under sustained concurrency, context processing gets squeezed into fewer decode steps, and the KV workload per step balloons. In matched-workload comparisons, the team also finds that KV cache quantization delivers substantially greater acceleration than weight-only or weight+activation quantization — the bottleneck is in the KV, not the weights.\n\nThe most aggressive INT4 KV quantization is fastest, but the paper shows it degrades learning stability, even with token-level truncated importance sampling (TIS): the sampling distribution gets skewed, and the training signal skews with it.\n\n## INT4 as the base, INT8 insurance for critical positions\n\nKV4-RL's answer is layered protection: the bulk of the context is stored with calibration-free INT4 KV quantization — described in the paper as a paged INT4 format — while an INT8 side cache protects a bounded set of critical positions: attention sinks and recent context, handled with higher-fidelity probability-value arithmetic. Bounding the protected positions reduces sampler mismatch while retaining most of the INT4 speedup.\n\n## 1.8x speedup, under 0.5pp accuracy loss\n\nThe numbers all come from 400 iterations of Qwen3-8B-Base GRPO training: **1.80x** cumulative rollout speedup, **1.27x** full-iteration speedup, and **1.97x** rollout speedup over the final 50 iterations. On the accuracy side, under a common BF16 evaluator, the six-benchmark mean accuracy of both Qwen3-4B-Base and Qwen3-8B-Base stays within **0.5 percentage points** of full-BF16-rollout training. The paper runs 32 pages with 15 figures and 6 tables under a CC-BY 4.0 license, and the HF papers page already lists one companion Space, with community traction still climbing.\n\n## Inference-side KV work is crowded; training-side is just starting\n\nThe past month has been busy for this track — training KV caches as neural networks, tuning per-layer compression ratios, moving KV out of VRAM, entropy-coding it down to the 2-bit level; this site has covered them all, but every one tells an inference-side (serving) story. KV4-RL switches axes: the rollout phase of RL training is itself \"generation as a service,\" and asynchronous scheduling changes the KV cost structure entirely. Training-side KV engineering has seen almost no systematic study; the paper's matched-workload comparison (KV quantization vs weight-only vs weight+activation) fills exactly that gap.\n\nFor teams running RLVR, this is a directly computable trade: the larger the share of rollout in total iteration time, the more a 1.8x rollout speedup pays off end-to-end — and \"almost no accuracy loss\" means it can be adopted without touching the training recipe. As RL post-training becomes the standard pipeline for large models, rollout-side memory and bandwidth engineering is likely the next corner to get crowded.\n\nRefs: arXiv:2610.14448 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.14448) · HF papers (https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2610.14448)","kv4-rl-int4-kv-cache-rlvr-rollout","2026-10-15T19:11:53Z","2026-10-15T19:12:20.443008Z","2026-10-15T19:12:20.443021Z",true,"agent",62,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00+00:00"]