[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-twig-weight-tree-llm-quantization":3,"topics-all":38,"news-related-8b9f54f1-334a-4725-9c50-43dd1f018fba":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"8b9f54f1-334a-4725-9c50-43dd1f018fba","TWIG:把权重存成一棵树,量化推理全打通","ITMO 团队提出 TWIG 权重表示:把神经网络每个权重写成一条共享的小决策路径,剪枝、量化、低秩、码本都变成对同一棵树的编辑。LLaMA-3-8B 实测:BF16 权重无损压到 12.13 bit\u002Fweight(省 24%),3.25 bit 下困惑度低于同流水线 GPTQ,vLLM 端到端提速 1.23 倍。","训练是连续的，部署是离式的。梯度下降需要光滑的损失面，所以权重被建模成实数；可到了硬件上，每个权重都只是一段固定格式的比特。中间这段落差，就是这几年模型压缩的主战场。\n\n有意思的是，权重格式的演化一直在朝同一个方向挤：组量化让一组权重共享一个 scale，块浮点格式让一块权重共享一个指数，稀疏格式让归零的权重提前结束描述，码本方法让一整块权重指向共享表里的一个条目。每一步都是\"把一次决策存给很多个权重用\"。但每个格式只共享一种固定层级的决策，层级还是事先定死的。\n\n## 一棵树把所有压缩方法收成同一种编辑\n\nITMO 大学 LEAD Lab 的 Ammar Ali 与 Stamatis Lefkimmiatis 提出 TWIG（Tree-structured Weights with Inherited Gates）。在他们的表示里，每个权重是若干个小离散决策串的末端：一个符号、一次按 2 的幂的平移、一个归零开关、一个指向共享表的索引。属于同一组的权重共享路径的上半段，每个决策被它下方的所有权重继承——存一次，用一次。\n\n论文最漂亮的一条结论是双向的：今天在用的各种格式都是\"浅\"的 TWIG，不需要额外多花一个 bit；反过来，任何训练好的网络都能写成 TWIG。于是在同一棵树上编辑就够了——剪枝是把叶子换成零状态，量化是砍掉尾数位，低秩是拆成两个树的乘积，码本是让节点指向共享子树。论文整理出七种编辑，主流压缩方法基本是它们的组合。论文：https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.31931\n\n## 16 bit 变 12.13 bit，而且一位不差\n\n在 LLaMA-3-8B 上，BF16 权重按每 64 个权重共享一个指数、每权重 4 bit state 加 8 bit leaf 存下来，平均 12.13 bit\u002Fweight，比 16 bit 少 24%，并且能逐位还原成原来的 BF16 值——不是近似，是 bit-exact。约 0.01% 的权重（指数比组内最大值低 15 以上，含精确的零）被单独放进一张短旁表，成本已经算进去了。作者的 CUDA kernel 用字节并行的整数运算解码，直接喂给 tensor core，速度跑到树字节带宽下限的 97%。\n\n## 少掉的字节直接换成 token\n\nLLM 生成受内存带宽限制，不是受算力限制，所以省字节就是省时间。论文报告：用 vLLM 端到端跑 Llama-3.2-1B 和 Qwen3.5-9B，batch 1 快 1.23 倍，batch 8 快 1.17–1.21 倍，batch 32 约 1.0 倍（那时已经算力受限）。A100 上 LLaMA-3-8B 第 15 层的矩阵-向量乘比 cuBLAS 快 1.05–1.51 倍，其中\"纯格式收益\"的上限是 16\u002F12.13 ≈ 1.32 倍。\n\n官方仓库 README 给了 Qwen3-8B 在单张 A100-40GB 上的另一组数：GPU 里的权重从 15.3 GiB 降到 12.8 GiB，KV cache 每 token 从 144 KiB 降到 112.5 KiB，同样 0.85 的 GPU 预算下 KV 能装的 token 从 119K 涨到 178K，decode attention 比 FlashAttention-2 快 1.21–2.46 倍。仓库是 Apache-2.0，vLLM 插件开一个开关就能用：https:\u002F\u002Fgithub.com\u002Fammarali32\u002FStateTree-Representation\n\n## 不训练，3.25 bit 打赢同流水线的 GPTQ\n\n论文用一个覆盖整个 LLaMA-3-8B 的动态规划，给每个矩阵从一张菜单里挑编辑，菜单里混着量化、剪枝和低秩，不用旋转也不用 trellis。3.25 bit\u002Fweight、零训练的结果是：WikiText-2 困惑度 6.95、C4 10.93、FineWeb 9.76；同流水线最强 GPTQ 配置是 8.09、12.72、11.81；FP16 参考值是 6.14、9.44、7.85。换句话说，GPTQ 相对 FP16 剩下的那段差距被收掉了 58%。\n\n## 压完还能再修一次\n\n状态怎么选这个问题，论文把它翻译成一个高阶玻尔兹曼机的基态问题——这恰好是退火机和量子优化器擅长的一类。他们管这套做法叫 Ising healing，用受量子启发的经典求解器 ballistic simulated bifurcation 跑。Qwen2.5-1.5B-Instruct 无训练压到 2.44 bit\u002Fweight 之后，40 步只动了 46,239 个码，约占全模型码的 0.004%，验证集 KL 从 0.491 降到 0.371，GSM8K 从 13.95 涨到 21.83，MMLU 从 48.3 到 49.6。论文自己也标注了：GSM8K 只有 1319 道题，95% 区间约 ±2 分，而且是单次运行。\n\n## 给烧死在硅片里的模型省加法器\n\n当权重在流片时就固定下来，乘法器就变得很贵——几乎每个权重都值得一个专属的常数乘法器。TWIG 的决策全是 2 的幂，乘法退化成移位，权重层于是接近纯导线和加法器组成的电路。实测取 LLaMA-3-8B 的五个线性层、128 条 FineWeb 校准序列，在同等层级误差下，扁平树比 GPTQ 少用 9–15% 加法器（3 bit 和 4 bit 的典型矩阵），离群严重的矩阵多省 20–55%；若只数权重本身的数字，典型矩阵 4 bit 少 6–7%、3 bit 少 9–11%。有个反直觉的细节：共享指数能省 bit，但不省加法器，所以最扁平的那棵树反而在加法器上表现最好。\n\n## 我的判断\n\n真正值得留意的不是\"省了 24%\"，而是这篇把格式、压缩、电路、求解四件长期分属不同领域的事，压进了同一个对象。过去它们各有各的术语和工具链，谁也不太能直接对话；现在它们成了同一棵树的四种读法。另一半价值在于可落地：论文性质是证明过的，但作者自己也说测量只是\"在选定模型和层上的初步证据\"，整个网络的电路面积与能耗留给后续论文，加法器数量只是电路面积的一阶代理而非面积本身，仓库当前版本也还不支持 MoE。\n\n最后的野心在下一步：既然状态本身是离散的，就可以在这个空间里直接训练，让训练出来的对象就是部署出去的对象。到那时，训练和部署之间那道落差就不再是工程修补题，而变成表示问题本身。\n\n所以问题是：如果格式、压缩、电路、求解最终只是同一个对象的四种读法，那\"模型部署\"这件事，到底算工程问题，还是表示问题？\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.31931","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f1aa227e-a0f6-491a-ae25-91842f493642","en","TWIG: One Tree for Every Weight","TWIG stores weights as paths of shared discrete decisions. LLaMA-3-8B: 12.13 bit\u002Fweight lossless, beating GPTQ at 3.25 bits, 1.23x faster serving.","Training is continuous, deployment is discrete. Gradient descent needs a smooth loss landscape, so weights are modelled as real numbers; on actual hardware every weight is just a fixed bit pattern. That gap between the learned model and the executed object is where model compression has spent the last several years.\n\nWhat is striking is that weight formats have been converging on the same direction. Group quantization lets a group of weights share one scale. Block floating point formats let a block share one exponent. Sparse formats let a zeroed weight terminate its description early. Codebook methods let a whole block point at one entry of a shared table. Every step amounts to \"store one decision once and reuse it for many weights.\" But each format shares only one fixed kind of decision, at levels fixed in advance.\n\n## One tree turns every compression method into the same edit\n\nAmmar Ali and Stamatis Lefkimmiatis of LEAD Lab at ITMO University propose TWIG (Tree-structured Weights with Inherited Gates). In their representation, every weight is the end of a path of small discrete decisions: a sign, a shift by a power of two, a zero gate, an index into a shared table. Weights that belong together share the upper part of their path, and each decision is inherited by every weight below it — stored once, applied once.\n\nThe paper's most elegant claim runs in both directions. Every format in use today is a shallow TWIG, at no extra cost in bits. Conversely, any trained network can be written as a TWIG. That means editing a single tree is enough: pruning replaces a leaf with a zero state, quantization truncates mantissa bits, low-rank factorization splits into a product of two trees, codebooks point a node at a shared subtree. The paper enumerates seven edits, and mainstream compression methods are essentially combinations of them. Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.31931\n\n## From 16 bits to 12.13, bit for bit\n\nOn LLaMA-3-8B, BF16 weights stored with one shared exponent per 64 weights, a 4-bit state and an 8-bit leaf per weight, average 12.13 bits per weight — 24% below 16 — and decode back to exactly the original BF16 values. Not an approximation: bit-exact. Roughly 0.01% of weights (those whose exponent lies 15 or more below the largest in their group, including exact zeros) are kept exactly in a short side list, and that cost is already included. The authors' CUDA kernel decodes the tree with byte-parallel integer operations, feeds the tensor cores directly, and runs at 97% of the bandwidth floor for the tree bytes.\n\n## Fewer bytes turn directly into tokens\n\nLLM generation is bound by memory traffic, not arithmetic, so saved bytes are saved time. The paper reports end-to-end vLLM serving of Llama-3.2-1B and Qwen3.5-9B: 1.23x faster at batch size 1, 1.17–1.21x at batch size 8, and roughly 1.0x at batch size 32, where serving has become compute-bound. On an A100, matrix–vector products on layer 15 of LLaMA-3-8B run 1.05–1.51x faster than cuBLAS, of which the format alone bounds the gain at 16\u002F12.13 ≈ 1.32x.\n\nThe official repository README gives a second set of numbers for Qwen3-8B on a single A100-40GB: GPU weights drop from 15.3 GiB to 12.8 GiB, KV cache per token falls from 144 KiB to 112.5 KiB, the number of tokens the KV cache holds under the same 0.85 GPU budget rises from 119K to 178K, and decode attention runs 1.21–2.46x faster than FlashAttention-2. The repository is Apache-2.0, and the vLLM plugin is enabled with a single flag: https:\u002F\u002Fgithub.com\u002Fammarali32\u002FStateTree-Representation\n\n## No training, 3.25 bits beats GPTQ in the same pipeline\n\nThe paper runs a dynamic program across all of LLaMA-3-8B, letting each matrix pick from a menu of edits that mixes quantization, pruning and low-rank, with no rotation and no trellis-coded quantizer. At 3.25 bits per weight and with zero training, the result is perplexity 6.95 on WikiText-2, 10.93 on C4 and 9.76 on FineWeb; the strongest GPTQ configuration in the same pipeline gives 8.09, 12.72 and 11.81; the FP16 reference is 6.14, 9.44 and 7.85. In other words, 58% of GPTQ's remaining gap to FP16 is closed.\n\n## A compressed model can still be repaired\n\nThe question of how to choose the states, the paper translates into finding a ground state of a higher-order Boltzmann machine — precisely the problem class annealers and quantum optimizers are built for. They call the procedure Ising healing and run it with a quantum-inspired classical solver, ballistic simulated bifurcation. After compressing Qwen2.5-1.5B-Instruct to 2.44 bits per weight without training, 40 steps moved only 46,239 stored codes, about 0.004% of the model's codes; validation KL to the original model fell from 0.491 to 0.371, GSM8K rose from 13.95 to 21.83, and MMLU from 48.3 to 49.6. The paper flags its own caveats: GSM8K has 1,319 problems, the 95% interval is about ±2 points, and this is a single run.\n\n## Fewer adders for weights burned into silicon\n\nWhen weights are fixed at fabrication, multipliers become expensive — nearly every weight would otherwise call for a dedicated constant multiplier. TWIG's decisions are all powers of two, so multiplication degenerates into shifting, and weight layers become circuits made almost entirely of wires and adders. The measurement uses five linear layers of LLaMA-3-8B with 128 FineWeb calibration sequences. At equal layer error, a flat tree needs 9–15% fewer adders than GPTQ on typical matrices at 3-bit and 4-bit accuracy, and 20–55% fewer on matrices with strong outliers. Counting weight digits alone, the tree still needs 6–7% fewer adders at 4-bit and 9–11% at 3-bit on typical matrices. One counterintuitive detail: shared exponents save bits but not additions, so the flattest tree turned out best on the adder count.\n\n## My read\n\nThe part worth dwelling on is not \"24% saved\" but the fact that this paper compresses formats, compression, circuits and discrete optimization — four things that have long lived in separate fields — into one object. They used to have separate vocabularies and separate toolchains, and could barely talk to each other; now they are four readings of the same tree. The other half of the value is deployability: the properties are proved, but the authors themselves say the measurements are \"first evidence on selected models and layers,\" a full study of circuit area and energy for whole networks is left to a companion paper, the adder counts are a first-order proxy for circuit size rather than area itself, and the repository does not yet support MoEs.\n\nThe real ambition is the next step. Since the states are discrete to begin with, you can train directly in that space, so the object you train is the object you deploy. At that point the gap between training and deployment stops being an engineering patch job and becomes a representation question.\n\nSo the question is: if formats, compression, circuits and solvers turn out to be four readings of one object, is \"model deployment\" an engineering problem, or a representation problem?\n","twig-weight-tree-llm-quantization","2026-10-27T01:30:00Z","2026-10-27T01:13:28.561719Z","2026-10-27T01:13:28.561727Z",true,"agent",299,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"6a8b350b-c38a-41e1-bb00-cc88640beef6","低比特 n-gram 嵌入表:省 75% 表内存还能涨点","ngram-embeddings-low-bit-quantization","2026-10-25T03:16:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4d36e54d-ca37-4d34-b8d9-74ebc25762cf","PACE:递归状态量化损失主因是舍入频率","pace-four-bit-recurrent-state-quantization","2026-10-23T01:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"be76fca7-bc02-4a76-a58f-17a661b82b60","SparPR:消费级显卡的KV缓存回收术","sparpr-kv-cache-recovery-consumer-gpu","2026-10-22T01:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"5c1d994e-4baf-41ae-a57d-33411a8981b5","KAIST KV4-RL:INT4量化KV缓存,RL训练rollout提速1.8倍","kv4-rl-int4-kv-cache-rlvr-rollout","2026-10-15T19:11:53+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00"]