[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ollama-official-tags-non-functional-quantized-llm-audit":3,"topics-all":38,"news-related-8db153fd-969d-4d3f-bd34-11304a3806d2":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"8db153fd-969d-4d3f-bd34-11304a3806d2","Ollama 官方库 7 个标签是坏的,文件本身却没坏","一份普查把 Ollama 官方库的 305 个代码类 GGUF 标签全部拉下来实跑,发现七个官方标签在 164 道基准题上几乎无法工作,占全部标签的 2.3%。重建原始权重后逐张量比对显示文件本身没坏,问题恰恰出在缺少重要性矩阵校准这一步;补上后 3B 编码模型的 q2_K 量化从零题恢复到 106-113 题。","现在把开源模型拉到本地跑，几乎都走同一条路：从注册表下载一个已经量化好的文件直接用。GGUF 是 llama.cpp 的单文件格式，Ollama 就构建在它上面，那些标签是每个人拿到模型的第一站。\n\n但这些文件下载下来，还真的能用吗？\n\n一份 10 月 29 日更新到第三版的普查（[论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05881)）给出了不太好看的答案。\n\n## 一次覆盖 305 个标签的普查\n\n作者把 Ollama 官方模型库 15 条模型线里所有符合条件的代码类 GGUF 标签全拉了下来——305 个，其中 291 个是量化版本——再加上 Hugging Face 上 22 个热门社区文件，一共执行 327 个产物。\n\n筛查分两步：先用 15 道代码题挑出可疑对象，再把这批对象拿到全部 164 道 HumanEval+ 任务上重跑。\n\n结果：7 个官方标签完全跑不动，占 305 个的 2.3%，涉及 35 个模型变体里的 3 个。其中 6 个在 164 道题上一道都过不了，剩下的 llama3.2-1B q3_K_S 也只过了 6 道。这 7 个全是参数不超过 40 亿模型上的低比特量化版本。\n\n作者还标注了一个下限：全量跑下来，在非可疑对象里又找出 8 个只解出 16 道题以内的产物。所以 2.3% 是下界。\n\n## 关键反转：文件没有坏\n\n怀疑者自然会往文件损坏上想。作者的做法是拿到原始 checkpoint，按文件元数据写的配方自己重量化一遍，逐张量对比。\n\n四个 Qwen 标签在全部 434 个张量上与官方文件一致，phi3.5 那个在 197 个上一致，15 道题上两边输出逐字节相同。文件没被污染，也不是传输损坏。\n\n论文原话：精确可复现并不等于功能正常。\n\n三个更简单的解释被逐一排掉：对话模板——校准后的构建共用同样的模板，照样能跑；运行时——CUDA 和 Metal 上都复现，换机器、换更新的 Ollama 版本也复现；权重绑定——能正常工作的 1.5B 就是绑定的。\n\n## 差的是校准这一步\n\n补上重要性矩阵（llama.cpp 里的 imatrix）后，权重、工具链、张量布局全部不变，结果是：Qwen2.5-Coder-3B 的 q2_K 从 0 题变成 106–113 题，q3_K_M 从 0 变成 76–89，q3_K_L 从 0 变成 80–86。第三方 bartowski 的构建给出 104、81、77，Qwen 自己的 AWQ 文件是 102 和 125，彼此吻合。\n\n但 q3_K_S 怎么校准都只有 13–30 题，救不回来。\n\n更值得琢磨的是尺寸差：同一套没校准的配方用在 1.5B 上完全正常，338 个张量一致，15 道题能过 13–15 道。配方本身预测不了失败，尺寸才是关键变量。\n\n## 不执行生成代码也能筛出来\n\n困惑度是个便宜代理：五个被测出的不可用构建，WikiText-2 困惑度是各自 fp16 版本的 27 到 5,700 倍，而 4-bit 标签还在 fp16 的 5% 以内（4.2% 和 5.1%）。阈值设在 10 到 25 倍之间，不执行任何生成的代码就能把这五个全标出来。官方库 q2_K 那个文件的困惑度是 66,391。\n\n代价也清楚：每个产物约 30 秒生成，66 个通过筛查的抽样标签里没有一个是非功能性的。工具已经放出来，叫 quantcheck。\n\n## 同一堵墙，两个独立来源\n\n这不是新现象。llama.cpp 仓库 2025 年 3 月的讨论 #12378 里，有人报告把 Qwen2.5-3B-Instruct 转成 Q2_K，在两台机器上输出都是胡言乱语——[这条讨论](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fdiscussions\u002F12378)至今没有解答。\n\nSingh 等人的独立审计（arXiv:2610.04403，10 月 3 日提交）从另一个方向撞上同一堵墙：13 个家族、27 个量化产物，参数量 0.35B 到 14B，一路压到 Q2_K，用 429 个经频率校验的罕见英文词做探针。Q2 档出现三种命运——彻底塌掉不可用、子 2B 模型语义解离、3B 以上基本完好。破坏程度主要由参数量决定（rho=0.72），词表大小完全预测不了（rho=0.12）。\n\n两边有一处张力值得留着：本文主张用困惑度倍数当廉价门，而 Singh 那份发现 WikiText-2 困惑度标定不了词义损伤——Q2 惩罚几乎一样（44.7% 和 47.6%）的两个产物，一个还保着词义（只掉 3.6%），另一个掉 43.6%。困惑度能筛出「塌了」，筛不出「说得出、但已经不懂在说什么」。\n\n## 所以呢\n\n论文引用了两项测量：全网扫描到超过 320,000 个公网可达的自托管 LLM 服务，其中 155,423 个是 Ollama 实例；另一项为期一年的测量记录到 152,137 个暴露 Ollama 的独立 IP。这套生态的默认假设一直是「下载即用」。\n\n这次普查说明中间少了一环验收。30 秒生成筛查加一个重要性矩阵，就能把不可用标签挡在发布之前；论文的建议正是：低比特标签发布前先筛查，失败的校准或者不下发。\n\n至于你自己——从 Ollama 拉下来的模型如果输出一片胡话，先别急着怀疑自己的显卡。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05881","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"1aa5ab09-742f-40f0-ac12-ecddb0989c08","en","Seven Ollama Tags Are Broken, and the Files Are Fine","An audit of 305 Ollama GGUF tags found 7 (2.3%) non-functional. Rebuilds matched originals tensor for tensor: the missing step was calibration.","Running an open-weight model locally usually starts with one step: pull a pre-quantized file from a registry and use it. GGUF is the single-file format of llama.cpp, the inference engine Ollama is built on, and those registry tags are the first thing most people touch when they install a model.\n\nBut do those files actually still work?\n\nA census that reached its third revision on October 29 ([the paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05881)) gives an uncomfortable answer.\n\n## A census covering 305 tags\n\nThe authors pulled every eligible code-capable GGUF tag from 15 model lines in the official Ollama library — 305 tags, 291 of them quantized — plus 22 popular community files on Hugging Face, for 327 artifacts executed in total.\n\nScreening ran in two stages: a 15-task code screen flagged suspects, which were then rerun on all 164 HumanEval+ tasks.\n\nSeven official tags turned out to be non-functional. That is 2.3% of the 305 official artifacts, spanning 3 of the 35 model variants. Six of them solved none of the 164 tasks; the remaining one, llama3.2-1B q3_K_S, solved 6. Every one of the seven was a low-bit quantization of a model with at most four billion parameters.\n\nThe authors also flag their own lower bound: running the full suite turns up eight more artifacts among the non-suspects that solve at most 16 of 164. So 2.3% is a floor, not the total.\n\n## The twist: the files are not corrupted\n\nThe obvious suspicion is a damaged download. The authors instead downloaded the original checkpoints, requantized them using the recipe recorded in each file's metadata, and compared tensor by tensor.\n\nThe four Qwen tags matched the official files across all 434 tensors; the phi3.5 tag matched across 197; and both sides produced byte-identical answers on all 15 tasks. Nothing was corrupted in transit.\n\nThe paper's phrasing: exact reproducibility did not imply function.\n\nThree simpler explanations are ruled out one by one. Chat templates: the calibrated builds share the same template and work fine. Runtime: the failure reproduces on CUDA and Metal, on a separate machine, and on a later Ollama release. Tied embeddings: the working 1.5B model ties them too.\n\n## What is missing is calibration\n\nAdd an importance matrix (the imatrix in llama.cpp) and, holding weights, toolchain and tensor mix fixed, the numbers move: Qwen2.5-Coder-3B q2_K goes from 0 to 106–113 of 164 tasks, q3_K_M from 0 to 76–89, q3_K_L from 0 to 80–86. Third-party bartowski builds agree at 104, 81 and 77, and Qwen's own AWQ files score 102 and 125.\n\nBut q3_K_S stays weak under every calibration, at 13–30 of 164. It does not come back.\n\nThe size contrast is the more interesting part. The same uncalibrated recipe applied to the 1.5B model stays functional — 338 tensors matching, 13–15 of 15 tasks passing. The recipe alone does not predict failure; size does.\n\n## Screening without executing generated code\n\nPerplexity is a cheap proxy: each of the five non-functional builds measured has 27 to 5,700 times the perplexity of its fp16 model, while the 4-bit tags stay within about 5% of it (4.2% and 5.1%). A threshold between 10 and 25 times fp16 flags all five without running any generated code.\n\nThe cost is bounded too: roughly 30 seconds of generation per artifact, and none of the 66 sampled tags that passed the screen was non-functional. The tool is out, named quantcheck.\n\n## The same wall, from two independent directions\n\nThis is not new. In a llama.cpp discussion from March 2025, someone reported that converting Qwen2.5-3B-Instruct to Q2_K produced nonsense on two different machines; [that thread](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fdiscussions\u002F12378) still has no answer.\n\nAn independent audit by Singh et al. (arXiv:2610.04403, submitted October 3) hit the same wall from another direction: 27 quantized artifacts across 13 families, from 0.35B to 14B parameters, pushed down to Q2_K, probed with 429 frequency-validated rare English words. At Q2 three fates emerge — total collapse into unusable builds, severe semantic dissociation in sub-2B models, and mostly robust preservation above about 3B. Parameter count dominates the damage (rho=0.72); tokenizer vocabulary size does not predict it at all (rho=0.12).\n\nOne tension is worth keeping. This paper proposes perplexity multiples as a cheap gate; the other audit finds WikiText-2 perplexity does not calibrate semantic damage — two artifacts with near-identical Q2 penalties (44.7% and 47.6%) split cleanly, one keeping its definitions (down 3.6%) and one losing them (down 43.6%). Perplexity can catch what has collapsed; it cannot catch what still speaks fluently but no longer knows what it is saying.\n\n## So what\n\nThe paper cites two measurements: an internet-wide scan found more than 320,000 publicly reachable self-hosted LLM services, including 155,423 Ollama instances; a year-long measurement saw 152,137 distinct IP addresses exposing Ollama. The ecosystem's default assumption has always been download-and-run.\n\nThis census shows a missing acceptance step. A 30-second generation screen plus an importance matrix can keep unusable tags out of the registry before publication; the paper's own recommendation is to screen low-bit tags before publishing, then calibrate or withhold the failures.\n\nFor your own setup: if a model pulled from Ollama emits gibberish, do not assume your GPU is at fault.\n","ollama-official-tags-non-functional-quantized-llm-audit","2026-10-31T03:15:00Z","2026-10-31T03:12:44.213548Z","2026-10-31T03:12:44.213559Z",true,"agent",24,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"d3807737-6e24-41cc-9012-9d7aeb5eed22","RL 训完不如基座:FORK 用精确率改写评分标准奖励","fork-rubric-rl-precision-reward","2026-10-27T05:10:19+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"dcc6132a-e432-4ef3-9555-5412d692cd90","RSIHub 开源:先冻结评测器,再谈自我进化","rsihub-agent-evolution-frozen-evaluator","2026-10-26T07:20:28+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"aa718f3c-14cf-4984-8762-794eaaee8c16","MAI-Gym:9B 智能体在 2 万个并行环境里做在线 RL","mai-gym-gui-agent-online-rl-9b","2026-10-21T09:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ab1e6bdf-9433-41f0-be4d-cc0f78523ac2","SonicBoom v0.0.3:一个 .so 干掉手工调优","sonicboom-v003-mlir-compiler","2026-10-19T09:03:05+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"bb6e0f7b-3899-4e45-a267-bfa512d470b0","FDE-Gym:95个真实交付任务,最强Agent只过14个","fde-gym-agent-delivery-benchmark","2026-10-18T23:08:06+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d529114c-bfd7-4efb-9af4-444d3065b397","Step 5 Preview 开源,2.7 美元\u002F百万输出","stepfun-step-5-preview-600b-moe","2026-10-17T21:12:04+00:00"]