[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-xor-trellis-hadamard-free-fp4-llm-quantization":3,"topics-all":35,"news-related-5d50bea3-d06f-45dc-982d-56d6e38eb45e":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"5d50bea3-d06f-45dc-982d-56d6e38eb45e","XOR-Trellis 把 4-bit 量化「去 Hadamard」:Llama-3.1 PPL 8.00","Arm 团队把 LLM 4-bit 量化的两条行业路线并行——XOR 解码器+曲率感知搜索,不再用 Hadamard 抹权重分布。Llama-3.1-8B WikiText-2 困惑度 8.00,优于启用了 Hadamard 的 QTIP 基线 8.48,且 4-bit 反量化可扛 GPU 权重吞吐。","把 LLM 压到 4-bit 以下的难点,从来不在「能不能压」,而在「压完之后能不能跑得快」。当前两条主流路线——基于 Hadamard 旋转的 incoherence processing(QTIP、SpinQuant 等),和基于结构化码本的 trellis-coded quantization(TCQ)——各解决了半个问题:Hadamard 让权重分布更均匀,从而让低比特量化器更容易拟合,但它无法被吸收进相邻算子,必须在推理路径里在线执行,挤占吞吐;Trellis 量化用 Viterbi 搜索 + 状态相关映射提供高维整形能力,代价是每一步都要按状态做反量化,如果每次反量化要算一堆逻辑\u002F算术\u002F数据操作,GPU 的权重消费速度跟不上,带宽红利就被吃光。\n\nArm 在 arXiv 公开的 **XOR-Trellis** 论文(arXiv:2610.00432,2026-09-30)同时拆掉了这两堵墙。它没有把 RHT 当成必备前置,而是把「Hadamard 让量化变好」这件事重新解释为:**RHT 之所以有效,主要是因为它把权重敏感度的不均匀抹平了,从而让传统的各向同性欧式距离恰好成为了模型感知目标的一个合理近似**。换句话说,真正缺的不是 RHT 本身,而是「搜索目标要反映模型敏感度」。论文的核心方法因此是两条独立但互补的设计:\n\n- **XOR-Trellis FP4 解码器**:用四组精心设计的局部重建调色板(P0 = [-6,-1.5,0,2]、P1 = [-4,-1,0.5,3]、P2 = [-3,-0.5,1,4]、P3 = [-2,0,1.5,6])配上一个 16-bit 滑动状态,通过四个掩码奇偶函数(0x1089\u002F0x2112\u002F0x0A24\u002F0x2441)XOR 出 p0,p1,q0,q1,前两位选调色板、后两位做分支置换。反量化从「状态→数值」变成「按位 XOR + 小型 FP4 调色板查表」,不再是动态查询或复杂计算函数,而是可以在 GPU\u002FNPU 上高并行执行。论文附录 C.2 证明这四组调色板在固定 FP4 元素、均匀使用四个互斥 4-项调色板、标量平方误差下,是 4-way 重建问题的点对点最优解。\n- **曲率感知 Viterbi 目标**:不旋转权重,而是把 LDL 分解 H = LDL⊤ 得到的对角因子 D 直接喂进 Viterbi 的加性失真代价。D 中的大对角项代表「该维量化误差对模型损失更敏感」,搜索过程中给这些位置更高的权重。整个目标在原始坐标空间里计算,完全离线、不修改压缩表示,运行时反量化器零开销。\n\n## 数字怎么说\n\n论文报告的 Llama-3.1-8B WikiText-2 困惑度(PPL,越低越好):\n\n- 未旋转的基线 trellis:8.69\n- 启用了 Hadamard 的 QTIP-3INST:8.48\n- XOR-Trellis(曲率感知、无 Hadamard):8.00\n\n也就是说,论文的方法既超过了「加 RHT」的传统方案,又彻底砍掉了在线 Hadamard 这条开销。\n\n跨模型扩展(完整数据见论文附录 B Tables 4–7):\n\n- Llama-3.2-1B-Instruct:BF16 13.16 → XOR-Trellis(无 RHT + D 加权 + 能量排序)21.38 PPL,零样本平均准确率从基线 46.74% 拉到 51.43%。\n- Llama-3.2-3B-Instruct:零样本平均从 54.03% 拉到 58.18%。\n- Llama-2-13B:5.47 PPL \u002F 65.38% 平均,略胜启用了 Hadamard 的 QTIP-3INST(5.64 \u002F 65.51%)。\n- Mistral-7B:5.67 PPL \u002F 67.95% 平均,大幅优于 QTIP-3INST 的 6.04 \u002F 66.80%。\n- Qwen3-8B:10.25 PPL \u002F 67.75% 平均,同样压过带 RHT 的 QTIP-3INST(11.35 \u002F 67.52%)。\n\n数字的一致性说明这个方法不是某个模型的「幸运调参」,而是实打实的结构改进。\n\n## 所以呢\n\nXOR-Trellis 的真正卖点不在 PPL 0.5 的提升,而在「Hadamard 是非必要步骤」这件事的正面论证。过去一年,行业围绕 RHT 和 learned rotation(QuaRot、SpinQuant)下了很多赌注,前提是「权重敏感度必须被某种旋转抹平才能压」。这篇论文用「对角 Hessian 加权 + 局部调色板 XOR」把前提打掉了:\n\n- 对硬件厂商来说,反量化路径上的计算变得可控——XOR + 4 项查表可以在 GPU 上做 SIMD,不再受制于动态计算函数。\n- 对模型侧研究者来说,「Hadamard 必要论」这个隐含假设可以休矣,可以把工程精力放在更难、更缺人的地方(比如激活量化、MoE 量化)。\n- 对 NVIDIA Blackwell\u002FFp4 GEMM、Rubin block-scaled FP4 这条硬件路线,论文的实验数据提供了软件层 Hadamard-free 的可行性证据,意味着未来 FP4 推理栈可以从「通用权重 → 旋转 → 量化」缩短为「通用权重 → 量化」。\n\n附带的另一个细节是分组格式:论文用 E2M1 元素 + E5M3 block scale,与 NVIDIA Blackwell\u002FRubin 的 block-scaled FP4 路线对齐——也就是说,论文方法不是一个理论玩具,而是对齐硬件路线的实战版本。\n\nLLM 量化这条赛道从 GPTQ(2022)走到今天,核心问题都是「压缩带来的精度下降,如何不用大代价翻回来」。XOR-Trellis 的贡献是:这条回边的成本,比行业以为的要低得多,而且可以完全在量化侧完成,不用 RHT 这种横跨整个推理图的预处理。下一步要看 Arm、这种 SMFP4 是不是能在 LLM 推理栈里被主流框架(vLLM、TensorRT-LLM、llama.cpp)集成进主路。参考资料:[arXiv:2610.00432](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.00432) 全文及附录 B 的 Tables 4–7。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.00432","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"35429e54-d859-4b3d-9a0e-005b6e448b7f","en","XOR-Trellis drops Hadamard from FP4 LLM quantization","Arm publishes XOR-Trellis: drop Hadamard, add curvature-aware Viterbi search. Llama-3.1-8B PPL hits 8.00, beating QTIP baseline 8.48.","Pushing LLMs below 4 bits has never been about whether compression is possible; it has been about whether the model can still run fast. Today's two industry routes—Hadamard-rotation-based incoherence processing (QTIP, SpinQuant, etc.) and trellis-coded quantization (TCQ) with structured codebooks—each solve half the problem. Hadamard makes the weight distribution more uniform, so low-bit quantizers can approximate better, but it cannot be absorbed into neighboring operators and therefore has to run online in the inference path, eating throughput. Trellis quantization gives high-dimensional shaping via Viterbi search and state-dependent mapping, but every step requires state-keyed dequantization; if each dequant needs a bunch of logic\u002Farithmetic\u002Fdata-movement operations, GPU weight-consumption bandwidth cannot keep up, so the bandwidth dividend is eaten up.\n\nThe **XOR-Trellis** paper (arXiv:2610.00432, 2026-09-30) from Arm removes both walls at once. Instead of treating RHT as a mandatory pre-step, it re-explains why Hadamard makes quantization better: **RHT is effective primarily because it smooths out non-uniform weight sensitivity, which makes conventional isotropic Euclidean distance a reasonable proxy for a model-aware objective**. In other words, what is missing is not RHT itself, but \"search objective must reflect model sensitivity\". The paper therefore proposes two independent but complementary designs:\n\n- **XOR-Trellis FP4 decoder**: uses four carefully designed local reconstruction palettes (P0 = [-6,-1.5,0,2], P1 = [-4,-1,0.5,3], P2 = [-3,-0.5,1,4], P3 = [-2,0,1.5,6]) combined with a 16-bit sliding state. Four masked-parity computations (0x1089\u002F0x2112\u002F0x0A24\u002F0x2441) XOR out p0, p1, q0, q1; the first pair selects the palette, the second permutes how the four branch identities map to palette entries. Dequantization collapses from \"state-to-value lookup\" to \"XOR + small FP4 palette lookup\"—no dynamic query, no expensive computed function, only operations that are highly parallel on a GPU\u002FNPU. Appendix C.2 proves the four palettes are pointwise optimal for the 4-way reconstruction problem under fixed FP4 elements, uniform use of four disjoint 4-entry palettes, and scalar squared-error distortion.\n- **Curvature-aware Viterbi objective**: rather than rotating the weights, it feeds the diagonal factor D from the LDL decomposition H = LDL⊤ directly into the additive distortion cost of Viterbi. Large diagonal entries in D represent dimensions where quantization error is more costly to the model loss, so the search gives those positions higher weight. The whole objective is computed in the original coordinate space, fully offline, without modifying the compressed representation, with zero overhead added to the runtime dequantizer.\n\n## What the numbers say\n\nOn Llama-3.1-8B WikiText-2 perplexity (PPL, lower is better), as reported in the paper:\n\n- Unrotated baseline trellis: 8.69\n- Hadamard-enabled QTIP-3INST: 8.48\n- XOR-Trellis (curvature-aware, no Hadamard): 8.00\n\nThat is, the paper's method beats the \"add RHT\" tradition and completely cuts the online Hadamard path.\n\nCross-model extension (full data in Appendix B Tables 4–7):\n\n- Llama-3.2-1B-Instruct: BF16 13.16 → XOR-Trellis (no RHT + D-weighted + energy ordering) 21.38 PPL, zero-shot average accuracy lifted from baseline 46.74% to 51.43%.\n- Llama-3.2-3B-Instruct: zero-shot average lifted from 54.03% to 58.18%.\n- Llama-2-13B: 5.47 PPL \u002F 65.38% average, slightly above the Hadamard-enabled QTIP-3INST (5.64 \u002F 65.51%).\n- Mistral-7B: 5.67 PPL \u002F 67.95% average, well ahead of QTIP-3INST's 6.04 \u002F 66.80%.\n- Qwen3-8B: 10.25 PPL \u002F 67.75% average, also beating the RHT-enabled QTIP-3INST (11.35 \u002F 67.52%).\n\nThe consistency across models shows this is not \"lucky tuning\" of one model but a real structural improvement.\n\n## So what\n\nXOR-Trellis's real selling point is not the 0.5 PPL bump, but the explicit argument that \"Hadamard is a non-essential step\". Over the past year, the industry has poured effort into RHT and learned rotation (QuaRot, SpinQuant) under the premise that \"weight sensitivity must be smoothed by some rotation before compression can work\". This paper removes that premise with \"diagonal Hessian weighting + XOR local palette\":\n\n- For hardware vendors, the dequant path becomes tractable: XOR + 4-entry lookup is SIMD-friendly on a GPU, no longer tied to dynamic computed functions.\n- For model-side researchers, the implicit \"Hadamard is necessary\" assumption can be retired; engineering effort can shift to harder and more underserved areas (e.g., activation quantization, MoE quantization).\n- For the NVIDIA Blackwell\u002FFP4 GEMM and Rubin block-scaled FP4 hardware roadmap, the paper provides software-layer proof of Hadamard-free viability, meaning future FP4 inference stacks can shorten from \"general weights → rotation → quantization\" to \"general weights → quantization\".\n\nOne more detail on the grouping format: the paper uses E2M1 elements + E5M3 block scales, aligning with the NVIDIA Blackwell\u002FRubin block-scaled FP4 roadmap—so this is not a theoretical toy but a battle-tested version aligned with hardware.\n\nThe LLM quantization track has run from GPTQ (2022) to today, with the core question always being \"how to recover the accuracy loss from compression without a big cost\". XOR-Trellis's contribution: the cost of that recovery edge is much lower than the industry assumes, and it can be done entirely on the quantization side, without RHT preprocessing that spans the entire inference graph. The next step is whether Arm\u002Fthis SMFP4 can be integrated into the main paths of mainstream LLM inference frameworks (vLLM, TensorRT-LLM, llama.cpp). Reference: [arXiv:2610.00432](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.00432) full text and Appendix B Tables 4–7.","xor-trellis-hadamard-free-fp4-llm-quantization","2026-10-05T04:00:00Z","2026-10-05T09:10:18.865638Z","2026-10-05T09:10:18.865651Z",true,"agent",134,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"645dd52f-ae6e-4931-9d74-4582feb4ecb1","Google TurboQuant：LLM推理内存压缩6倍的技术突破","google-turboquant-kv-cache-compression","2026-04-23T01:11:00+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"4ccc491b-dbee-4c84-beb6-7cf519f76320","LoRA 基座换 GGUF:40G 显存训 125B","lora-over-gguf-low-vram-training","2026-10-10T21:08:22+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"33f9c56d-1bcd-403d-85e2-273fb3d498fb","循环状态量化破局:STEPQuant 6-bit 贴平 FP32,显存省 68.7%","stepquant-delta-rule-state-quantization","2026-10-08T21:07:39+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"530e5aa5-2026-4c30-b4e6-421caca907b2","Transformer 提前罢工:13 个基座模型跟不住引用链,一个 rank-8 LoRA 修好","tiny-lora-frozen-transformer-chain-relay","2026-10-03T21:05:16+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"32aa44d5-81f1-4717-8d17-ff713101b725","Phonon-2:2.1比特量化ASR,164MB逼近全精度","phonon-2-2-bit-parakeet-asr","2026-10-03T13:10:31+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"f3b52d77-bfad-4289-adab-777d19a79bcb","Parakeet Redux把ASR压到178MB:纯CPU跑出113倍实时","parakeet-redux-ternary-cpu-asr","2026-10-01T21:08:38+00:00"]