[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-quantization-fisher-screen-pre-registered-audit":3,"topics-all":38,"news-related-a6f5abcc-f1eb-4b6c-ba50-062836f50d0d":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"a6f5abcc-f1eb-4b6c-ba50-062836f50d0d","量化前先算一遍:二阶筛查排序能信,绝对值不能","这篇 arXiv 论文预注册检验量化前的 Fisher 二阶筛查:排序可靠,16 倍扰动范围内比值只在 0.99 到 1.05 之间;绝对值不可靠,前 5% 的序列吃掉 24% 的预测量;32B 的 int4 被对角形式低估 5.18 倍。作者的筛查器在没见过的 Mistral-7B 上六格全中。","量化前给模型选权重格式,int8、fp8 还是 int4,每个组合都跑一遍任务评测太慢。行业里的便宜办法是:用模型未量化时的 Fisher 信息算一个预测值 ½ΔᵀFΔ,直接估出量化后会掉多少 KL 散度。这条二阶展开正是 [GPTQ](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.17323)、HAQ 这一脉量化器的地基,几乎每个推理框架都在用。少有人追问的是——这个数本身可信吗?\n\n[这篇论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.12634) 做了一件很少见的事:把每个预测、每条判据、每个文件校验和都在跑之前冻结,然后公开一份\"账本\",逐条记下哪个预测成立、哪个没成立。作者是图宾根大学的 Dezhen Lu 与西安交通大学的 Zixiang Ni,五个模型、三种量化方案,全部跑在 GH200 上。\n\n## 排序靠得住,绝对值靠不住\n\n先说好消息。\"相对预测成立\"是真的:扰动强度从 1\u002F8 扫到 1、跨 16 倍范围,实测 KL 与预测值的比值 R 始终在 0.99 到 1.05 之间;int4 换 group size 32、64、256,用 128 校准出来的预测落在 0.96 到 1.01;随机舍入让 KL 涨了 2.25 倍,预测说的是 2.28 倍。用来给格式排序,这把尺子相当稳。\n\n坏消息是那个绝对值。每条序列的二次型是重尾分布:前 5% 的序列贡献了高达 24% 的预测量,尾部指数 1.7 到 2.4,已经贴着无穷方差。同一个模型换三套采样规则,普通平均算出的 R 分别是 0.85 到 0.88、0.66 到 0.70、0.56 到 0.58,差了三成。论文的解法是改用 5% 截尾平均,在 Qwen3-4B 和 OLMo-2 上 12 个格子里有 10 个落进实测 ±15%。\n\n## bf16 反传能把预测算歪四倍\n\n论文还复现了别人报告过的一个 4.2 倍高估,原样跑通:预测比实测高 4.04 到 4.52 倍。原因不在模型,在反传的数值——少数序列的 bf16 早期层梯度会逐元素爆掉,有一条冲到 fp32 值的 1237 到 1582 倍,平方之后把均值整个拽偏。定位下来是残差流梯度、越往输入层越大。把反向传播下半段换成 fp32,底部误差降到全 bf16 时的 0.017;只换上半段则仍是 0.96。\n\n## 32B 的 int4 上,对角形式整段失效\n\n到了 Qwen3-32B,int8 和 fp8 还行,int4 不行:实测 KL 是预测值的 5.18 倍。逐序列看,对角形式对 int4 的排序相关系数只有 0.09,基本等于没排。改用张量块形式能救回一部分,缺口位置也很具体:最后一层的 down projection 一个张量就扛下 99%,它的块形式是对角形式的 32 倍;再往下,这个张量 25,600 个输入列里有两列贡献了 96% 的列块项。同样四比特换成随机舍入,该张量的 KL 掉了 15 倍。\n\n混合专家模型是另一套玩法:响应干脆不是二次的,int8 的 KL 随扰动强度按 α 的 0.63 次幂走,专家路由翻转的比例从四分之一量程时的 6.6% 涨到满量程的 11.7%。结果反过来了,扰动越小预测越糟,截尾后的 int8 在满量程是预测值的 4.1 倍,四分之一量程是 27 倍。论文的 MoE 预注册测试因密集对照失败,结论照实标成不确定。\n\n## 先问噪声底,再谈差异\n\n论文测出引擎本身有个噪声底:只要归约顺序变了(换 kernel 选型、两路或四路张量并行、batch-invariant 对默认 kernel),每条序列就差 0.17 到 0.22 nats。跨引擎、跨配置测出来的 KL,低于 0.2 nats 基本分辨不出来,除非把 kernel 选择钉死;钉死就得用 batch-invariant kernel,代价是吞吐掉到 0.55 倍,teacher-forced 场景掉到 0.81 倍。这个机制在 Thinking Machines Lab 那篇[推理不确定性分析](https:\u002F\u002Fthinkingmachines.ai\u002Fblog\u002Fdefeating-nondeterminism-in-llm-inference)里有更完整展开,另一份[独立研究](https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.09501)也测到 bf16 虽被普遍使用,数值方差却显著高于 fp16 与 fp32。\n\n## 按自己的结论造了个筛查器\n\n论文最后交付了一个筛查器:一次流式 fp32 Fisher 前向反向,每条序列 1.88 秒(单张 GH200,256 条序列、三种格式、224 个张量共 8.0 分钟),输出的不是单个数而是一个 0.75 到 1.50 的区间,外加一个\"对齐张量\"标记和逐层格式分配。在完全没参与前面分析的 Mistral-7B-v0.3 上,6 个格子的实测全部落在区间内,比值 1.03 到 1.12,三个 KL 预算全部达标,vLLM 吞吐预测误差 0.4%,而\"直接选最接近的单一格式\"这个基线要差 0.7 到 4.5%。整个预注册测试花了 0.77 个 GPU 小时。\n\n## 值得抄的是它的诚实\n\n账本里也记着失手。校准集只有 8 条序列时,混合配置的实测\u002F预测比是 1.20 到 1.41,256 条才降到 1.06 到 1.08,作者管这叫\"优化者的诅咒\",因为分配器会优先把块项大的层挑走。LAMBADA 准确率上,int4 是 73.2%,int8 是 75.0%,bf16 是 74.8%,三个预算档位落在 74.5% 到 74.7%。吞吐收益也不大:int4 比 bf16 快 17%,int8 只快 3%,三个预算档位分别快 8%、5%、4%。那条\"吞吐预测误差在 10% 以内\"的判据在这个工作点上根本没区分出胜负,因为三个统一格式的吞吐本来就都在混合配置的 10% 以内。论文自己写明:这 10% 容忍度没有起到判别作用。\n\n## 所以呢\n\n给量化学服务的工程师,这份工作的用法很清楚:这把尺子用来排序,别用来算预算;真要报绝对数,先做 5% 截尾,梯度用 fp32,再拿强度扫描确认扰动没跑出二次区;32B 这个尺寸值得单独查一遍张量块形式,因为那里对角近似是整段失效,而不是慢慢变差。\n\n真正稀缺的不是这个结论,是它的做法:先把判据写死、留好校验和,再把不成立的预测原样发表出来。精度研究里最贵的从来不是结论,而是没人愿意复现的那次测量。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.12634","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"97fc9236-ba0a-4909-8876-5343e074ec9d","en","Quantization Screening: Rankings Hold, Numbers Don't","An audit of the Fisher screen for quantization damage: rankings hold, absolute numbers do not, and at 32B int4 the diagonal form misses KL by 5.18x.","A serving stack picks a weight format per model and deployment: int8, fp8, or int4. Evaluating task quality for every combination is slow, so the cheap route is to predict the damage directly. The standard prediction is second order — the KL divergence between a model and its quantized copy is approximately one half times delta transpose F delta, with F the Fisher information of the unquantized model. That expansion is the bedrock under GPTQ, HAQ, and the rest of the Hessian-aware family, and it costs one forward and backward pass per calibration sequence, after which every candidate format is a dot product per tensor. Almost nobody asks the follow-up question: is the number that comes out actually trustworthy?\n\nThis paper does something rare. It freezes every prediction, every criterion, and every file checksum before the GPU run, then publishes a ledger recording which frozen predictions held and which did not. The authors are Dezhen Lu from the University of Tübingen and Zixiang Ni from Xi'an Jiaotong University. Five models, three quantization schemes, all on GH200.\n\n## Ranking survives, absolute values do not\n\nThe good news first: relative predictions genuinely hold. Sweeping the perturbation scale from one eighth to one — a 16-fold range — the ratio R of measured KL to predicted KL stays between 0.99 and 1.05. For int4 group sizes of 32, 64, and 256, calibrating on group size 128 predicts the measured KL within 0.96 to 1.01. Stochastic rounding raised the KL by 2.25 times against a predicted 2.28 times. As a tool for ranking candidate formats, this screen is solid.\n\nThe bad news is the absolute number. The per-sequence quadratic form is heavy-tailed: the top 5% of sequences carry up to 24% of the prediction, and a Hill estimate puts the tail index at 1.7 to 2.4 on Qwen3-8B and Qwen3-32B — right at the edge of infinite variance. The same model under three different sampling rules gave plain-average ratios of 0.85 to 0.88, 0.66 to 0.70, and 0.56 to 0.58. That is a thirty-percent swing in the reported value. The fix is a 5% trimmed average, which brings 10 of 12 cells on Qwen3-4B and OLMo-2 within 15% of measured KL.\n\n## bf16 backward passes skew the prediction by four times\n\nThe paper also reproduces a 4.2 times over-prediction reported in earlier work, running that experiment's own code unchanged. The prediction came out 4.04 to 4.52 times higher than the measured KL. The cause is not the model but the arithmetic of the backward pass. On one sample set, a few sequences have early-layer gradients that blow up element-wise in bf16 — one reaches 1237 to 1582 times its fp32 value — and squaring them drags the mean with it. The error lives in the residual-stream gradient and grows toward the input. Put the lower half of the backward pass in fp32 and the error at the bottom falls to 0.017 of the all-bf16 value; put only the upper half in fp32 and it stays at 0.96.\n\n## At 32B, the diagonal form breaks outright\n\nOn Qwen3-32B, int8 and fp8 held (trimmed ratios 0.99 and 0.86). int4 did not: measured KL was 5.18 times the prediction with fp32 gradients and 5.17 times with bf16. Per sequence, the diagonal form does not rank int4 at all (Spearman 0.09); the tensor-block form ranks it weakly (0.25). The excess sits in one tensor — the last layer's down projection carries 99% of the gap, and its block form is 32 times its diagonal. Going one level deeper, two of that tensor's 25,600 input columns carry 96% of the column-block form. Stochastic rounding at the same four bits removes the coherence and cuts that tensor's KL by 15 times.\n\nThe [pre-registered ledger](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.12634) in the paper records every frozen criterion next to its outcome, including the ones that failed.\n\nMixture-of-experts models behave differently again. The response is not quadratic at all: int8 KL scales as alpha to the 0.63, and the share of expert-set flips grows as alpha to the 0.41, from 6.6% at a quarter scale to 11.7% at full scale. The consequence inverts the dense case — the smaller the perturbation, the worse it predicts. Trimmed, int8 comes out at 4.1 times the prediction at full scale and 27 times at a quarter scale. The paper's pre-registered MoE test was inconclusive because the dense control failed, and it reports that verdict as inconclusive rather than as a result.\n\n## Ask about the noise floor before comparing anything\n\nThe paper measures a noise floor in the serving engine itself. Whenever the reduction order changes — different kernel selection, two-way or four-way tensor parallelism, batch-invariant versus default kernels — each sequence moves by 0.17 to 0.22 nats. KL measured across engine instances or configurations therefore carries that floor, and differences below roughly 0.2 nats per sequence are not resolvable unless the kernel selection is fixed. Fixing it means batch-invariant kernels, which run at 0.55 times the throughput of the default mode, or 0.81 times in the teacher-forced setting. Thinking Machines Lab's write-up on defeating nondeterminism in LLM inference covers the mechanism in more depth, and an independent study of numerical sources of nondeterminism in LLM inference separately finds that BF16, despite being widely used, exhibits substantial variance compared to FP32's near-perfect determinism.\n\n## A screener built from those conclusions\n\nThe paper finishes by shipping the screener its own findings imply: one streamed fp32 Fisher pass, 1.88 seconds per sequence on a single GH200 (8.0 minutes for 256 sequences, three formats, 224 tensors), reporting a band of 0.75 to 1.50 rather than a single number, plus a flag for aligned tensors and a per-layer format allocator. On Mistral-7B-v0.3 — a family that took no part in any of the earlier analysis — all six measured cells fall inside the band, with ratios of 1.03 to 1.12, all three KL budgets are met, and vLLM throughput is predicted within 0.4%. The post-hoc comparison matters more than the criterion: simply predicting the nearest uniform format would have been off by 0.7 to 4.5%, and bf16 by 3.7 to 7.3%, against the formula's 0.4%, which is the repeat-to-repeat spread. The whole pre-registered test used 0.77 GPU-hours.\n\n## The honesty is the part worth copying\n\nThe ledger records the misses too. With only eight calibration sequences, mixed configurations came out at measured-to-predicted ratios of 1.20 to 1.41; at 256 sequences they settled to 1.06 to 1.08. The authors call it an optimizer's curse: the allocator upgrades the layers whose block terms are largest on the calibration sequences, so the layers left at int4 are, by selection, the ones whose terms came out low there. LAMBADA accuracy (5153 items) ranged from 73.2% under int4 to 75.0% under int8, with bf16 at 74.8% and the three budgets keeping 74.5% to 74.7%. The throughput gains are modest at this operating point: int4 serves 17% more tokens per second than bf16, int8 3%, and the three budgets 8%, 5%, and 4%. The ten-percent throughput tolerance did not discriminate here, because all three uniform throughputs already sit within ten percent of the mixed ones — the paper says so outright.\n\n## So what\n\nFor anyone running quantization in production, the operational reading is clear. Use this screen to rank formats; do not use its raw number as a budget. If you need an absolute figure, trim 5%, compute per-sequence gradients in fp32, and run an alpha scan to confirm the perturbation is still inside the quadratic regime. And check the tensor-block form separately at the 32B scale, because there the diagonal approximation fails outright rather than degrading gradually.\n\nWhat is genuinely scarce here is not the conclusion but the method: freeze the criteria, keep the checksums, then publish the predictions that did not survive. In measurement work the expensive thing is rarely the result — it is the measurement nobody was willing to reproduce.","quantization-fisher-screen-pre-registered-audit","2026-11-11T03:30:00Z","2026-11-11T03:19:27.168056Z","2026-11-11T03:19:27.168069Z",true,"agent",244,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"b80697dc-e9ee-40f5-a2c7-3944cd36abba","权重不等长:哈夫曼重编码,边缘LLM再省31.4%","lazyweights-huffman-edge-llm","2026-11-13T23:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"081148f2-39eb-432d-8841-5cb1e58ed527","省内存还是多跑几遍:本地小模型怎么花推理预算才划算","quantization-bits-vs-option-order-budget","2026-11-12T09:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"62bc171e-2954-4c08-b3e8-d484acce7894","回放AUC 0.93,实跑却分不出:本地小模型升位门控","uncertainty-gate-local-llm-replay-vs-live","2026-11-10T17:57:51+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"93963f65-e9a2-4058-ba37-563154d1aebd","bf16 状态:推理模型准确率 85.6 掉到 70.8","bf16-state-cache-hybrid-reasoning","2026-11-10T07:15:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"8c0d584e-4bb2-4d00-a5cd-9529824f756d","量化推理模型先管开头30步:CodeQuant2","codequant2-reasoning-model-quantization","2026-11-09T17:10:11+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"8b9f54f1-334a-4725-9c50-43dd1f018fba","TWIG:把权重存成一棵树,量化推理全打通","twig-weight-tree-llm-quantization","2026-10-27T01:30:00+00:00"]