[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-temporal-rotation-2bit-kv-cache-video-diffusion":3,"topics-all":38,"news-related-cb1044d5-dc15-4f72-b720-e4a110f939e9":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"cb1044d5-dc15-4f72-b720-e4a110f939e9","论文实测:视频KV缓存2比特量化,主体一致性92.02","MIT运筹学中心提出TRQ,把视频扩散Transformer的KV缓存压到2比特:对相邻帧块在线拟合2×2旋转树,先摊平各通道共享均值再量化,不用梯度、不改权重。论文实测LongCat-Video-13B主体一致性从QVG的90.38升到92.02,压缩6.89倍,H100上单对KV旋转开销1.785毫秒。","长视频生成卡在哪?不在模型权重,在缓存区。视频扩散Transformer靠KV缓存复用历史帧的键值向量,省了重复计算,但缓存随生成历史线性增长,这篇11月19日挂上arXiv的论文的说法是,它能涨到数十GB,超过常规显卡装得下的范围 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26334)。QVG团队在另一篇论文里给的独立口径更具体:自回归视频生成的KV缓存经常超过30GB,直接挡住部署 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2602.02958)。MIT运筹学中心三名研究者的解法,是把这块缓存压到2比特,同时把主体一致性从90.38抬到92.02。\n\n## LLM的量化经验直接搬过来会翻车\n\nKV缓存量化在LLM侧已经成熟,QuaRot一路的思路是先旋转再量化:把离群通道混进正常通道,分布摊平了再压比特 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.00456)。但论文指出,这类方法是按LLM缓存的结构设计的,直接套到视频扩散Transformer上精度损失明显。视频缓存有自己的结构:同一个图像块在相邻几帧里的缓存向量,在每个通道上共享一个缓慢变化的均值。论文量化了这件事——按2帧一块切,Key缓存里91.22%的能量落在块的均值上;放宽到16帧一块,均值仍占76.88%。要压到2比特,均值就是必须先处理掉的大头。\n\n## 每块一组2比特角度,在线拟合\n\nTRQ的做法分两层。每个时间块(默认8帧)先配一棵2×2平面旋转的树:每个旋转的角度只从0、π\u002F4、π\u002F2、3π\u002F4四个值里选,所以每个角度只用2比特存储;块内所有帧共用这组旋转,树后面再接一个所有块共享的固定Hadamard变换。目标很直接:把块的逐通道均值旋成近似常数向量,让量化器的offset把共享成分整体吸收掉。整棵树用贪心法拟合,论文证明这个贪心拟合在它的均值摊平目标下就是最优解。关键在「在线」二字:每条新视频现拟合自己的旋转,不用梯度,不改模型权重。开销也压得够低——块大小8、通道数128时,一组旋转只要2048比特元数据。\n\n## 论文实测:数字站得住\n\n对比在LongCat-Video-13B、Self-Forcing-Wan-1.3B、HY-WorldPlay-8B三个视频扩散模型上跑,主战场是LongCat,100条配对视频、480p。同等压缩率下(TRQ 6.89倍,QVG 6.88倍),主体一致性:BF16基线93.02,QVG 90.38,TRQ 92.02;PSNR从24.06抬到26.53。720p的25条配对视频上差距更悬殊:93.46对91.06。8B的HY-WorldPlay,论文摘要报的是86.33升到87.61;论文并称在LongCat和Self-Forcing上七项指标全部领先、HY-WorldPlay上七中六。消融实验说明问题:只留Hadamard变换,PSNR从27.69掉到25.90;把学出来的角度换成随机的,再掉到23.30——在线拟合的角度才是增益来源。工程侧论文配了Triton内核,H100上每对K\u002FV的旋转开销从6.638毫秒压到1.785毫秒,整条缓存的处理时间从10.196秒降到2.743秒。\n\n## 该泼的冷水\n\n第一,论文自己承认,时间冗余弱的场景——快动作、镜头切换——是开放问题,自适应块长被列为后续方向。第二,TRQ走旋转路线,QVG走质心路线,论文明说两者结构互补、合并可能更强,但也只是指出,没有做。第三,论文称已发布融合Triton内核,但正文没有给出仓库链接,发布状态待核。第四,所有对比数字都是论文自报,被对标的QVG有多强,同样是论文的一面之词,还没有第三方复现。机制层倒是双源成立:QuaRot在LLM侧证明旋转能在不改输出的前提下压掉离群通道(LLaMA2-70B压到4比特,困惑度损失至多0.47,zero-shot保留99%),QVG在视频侧独立确认了30GB这堵墙。\n\n所以呢——长视频生成的下半场,拼的可能不是谁的模型更大,而是谁的缓存更省。为每条视频现配一组2比特的旋转,这条路论文已经走通;快动作和镜头切换那两个坑,就看谁先补上了。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26334","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",{"id":22,"name":23,"slug":23,"description":14,"color":14},"ebe5dcd1-46b1-4298-b8c2-8e0e2f456e56","video-generation",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"0588192f-c9c4-4546-a439-83704469e09c","en","Online temporal rotations lift 2-bit video KV cache to 92.02","MIT's TRQ quantizes video-diffusion KV caches to 2 bits with online per-block rotations. LongCat-13B subject consistency: 90.38 to 92.02 at 6.89x.","Long video generation is bottlenecked not by model weights but by the KV cache. Video diffusion transformers reuse cached keys and values across denoising steps, and the cache grows linearly with generation history; the paper, posted 19 Nov 2026, says it can reach tens of gigabytes, more than commodity GPUs can hold (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26334). An independent team behind QVG gives a sharper number: autoregressive video KV caches often exceed 30 GB, blocking deployment (https:\u002F\u002Farxiv.org\u002Fabs\u002F2602.02958). A three-author group from MIT's Operations Research Center now pushes that cache to 2 bits while lifting subject consistency from 90.38 to 92.02.\n\n## Why LLM recipes do not transfer\n\nKV-cache quantization is mature on the LLM side. The QuaRot line of work rotates first and quantizes second, mixing outlier channels into benign ones so the distribution flattens before bits are cut (https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.00456). The paper argues those methods were built for the structure of LLM caches and lose substantial accuracy when applied directly to video diffusion transformers. Video caches have their own structure: the cached vectors of one image patch across adjacent frames share a slowly varying per-channel mean. The paper quantifies it - with 2-frame blocks, 91.22% of the squared norm in Key caches sits in the block mean; even at 16-frame blocks the mean still holds 76.88%. Quantize that to 2 bits naively and the mean destroys everything.\n\n## Two-bit angles, fitted online per video\n\nTRQ gives each temporal block (default 8 frames) a tree of 2x2 planar rotations. Each angle is chosen from just four values - 0, pi\u002F4, pi\u002F2, 3pi\u002F4 - so it costs 2 bits of metadata; a fixed Hadamard transform shared by all blocks sits behind the tree. The goal is mean flattening: rotate the block's per-channel mean into a near-constant vector, so the quantizer offset absorbs the shared component. The fit is greedy, and the paper proves the greedy fit is globally optimal for this objective. Everything happens online for each new video, with no gradients and no weight changes. A block of 8 frames and 128 channels needs only 2048 bits of rotation metadata.\n\n## What the numbers say\n\nThe comparison spans LongCat-Video-13B, Self-Forcing-Wan-1.3B and HY-WorldPlay-8B, with the main arena on LongCat: 100 matched videos at 480p. At matched compression (TRQ 6.89x, QVG 6.88x), subject consistency reads 93.02 for the BF16 baseline, 90.38 for QVG, 92.02 for TRQ; PSNR rises from 24.06 to 26.53. On 25 matched 720p videos the gap widens: 93.46 versus 91.06. On the 8B HY-WorldPlay the paper's abstract reports 86.33 up to 87.61, and the paper claims leads on all seven evaluated metrics for LongCat and Self-Forcing, six of seven for HY-WorldPlay. The ablation tells the story: Hadamard alone leaves PSNR at 25.90, random angles sink to 23.30, versus 27.69 for the full method - the online-fitted angles are where the gain lives. On an H100, fused Triton kernels cut the per-pair rotation cost from 6.638 ms to 1.785 ms, and whole-cache processing from 10.196 s to 2.743 s.\n\n## The caveats\n\nFirst, the paper itself flags weak temporal redundancy - fast motion, scene cuts - as an open problem, with adaptive block lengths listed as future work. Second, TRQ (rotations) and QVG (centroids) exploit complementary structure; the paper says combining them could beat either alone, but does not do it. Third, the paper says fused Triton kernels are released, yet no repository link appears in the text. Fourth, every comparison number is self-reported, and QVG's standing is likewise the paper's framing, not an independent replication. The mechanism layer is on firmer ground: QuaRot showed rotations remove outliers without changing the output (LLaMa2-70B at 4 bits loses at most 0.47 WikiText-2 perplexity and keeps 99% of zero-shot performance), and QVG independently confirmed the 30 GB wall.\n\nSo the next phase of long video generation may be decided not by whose model is bigger but by whose cache is cheaper. Per-video, two-bit rotations are now a working path; fast motion and scene cuts are the holes to fill.","temporal-rotation-2bit-kv-cache-video-diffusion","2026-11-21T21:30:00Z","2026-11-21T21:11:32.195501Z","2026-11-21T21:11:32.195512Z",true,"agent",33,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"ff6f65e1-28b2-4a48-b317-7870072ecfa9","VC-Attention低比特注意力:视频生成提速1.59倍","vc-attention-low-bit-video-attention","2026-09-17T13:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"b80697dc-e9ee-40f5-a2c7-3944cd36abba","权重不等长:哈夫曼重编码,边缘LLM再省31.4%","lazyweights-huffman-edge-llm","2026-11-13T23:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"081148f2-39eb-432d-8841-5cb1e58ed527","省内存还是多跑几遍:本地小模型怎么花推理预算才划算","quantization-bits-vs-option-order-budget","2026-11-12T09:10:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"a6f5abcc-f1eb-4b6c-ba50-062836f50d0d","量化前先算一遍:二阶筛查排序能信,绝对值不能","quantization-fisher-screen-pre-registered-audit","2026-11-11T03:30:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"62bc171e-2954-4c08-b3e8-d484acce7894","回放AUC 0.93,实跑却分不出:本地小模型升位门控","uncertainty-gate-local-llm-replay-vs-live","2026-11-10T17:57:51+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"93963f65-e9a2-4058-ba37-563154d1aebd","bf16 状态:推理模型准确率 85.6 掉到 70.8","bf16-state-cache-hybrid-reasoning","2026-11-10T07:15:00+00:00"]