[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-btl-4-compact-9gb-2bit-quantization":3,"news-related-ce40a7fa-acca-4609-a82e-5800f2e1026a":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"ce40a7fa-acca-4609-a82e-5800f2e1026a","35B 压成 9.96GB 单文件:BTL-4 的 2.30 bit 极限量化和两条反直觉结论","Bad Theory Labs 开源的 35B MoE 模型 BTL-4 推出 Compact 版:整个模型压成 9.96GB 单文件、平均 2.30 bit\u002F权重,官方用重放测量给出 94.1% 行为保留率。model card 完整公开了量化配方与两条反直觉仿真结论:range selection 比什么都重要,而保护输出头毫无收益。","# 35B 压成 9.96GB 单文件:BTL-4 的 2.30 bit 极限量化和两条反直觉结论\n\n8 月 5 日,独立 AI 实验室 Bad Theory Labs 发布了两个开源权重模型:35B 的 agentic 推理模型 BTL-4,和跑在 Mac 上的本地智能体 Macaw。其中 BTL-4 的 Compact 版本把一件事做到了极致:**整个 35B 模型塞进一个 9.96GB 的文件,平均每个权重只占 2.30 bit,官方重放测量下保留 94.1% 的行为**。这份 model card 值得细读,因为它把 2-bit 级量化的完整工程账本摊开了。\n\n## 架构:大模型的内存,小模型的算力\n\nBTL-4 是 MoE 架构:总参数 35.1B,每个 token 只激活约 2.1B——官方形容它「花大模型的内存,花小模型的算力」。40 层里有 30 层线性注意力、10 层全注意力;每层 256 个专家、路由 8 个。原生上下文 262,144,而 KV cache 只在 10 层增长、约 20KB\u002Ftoken——整个 262K 窗口的缓存约 5.2GB。长上下文任务塞进消费级硬件,是设计目标而非巧合。\n\n## 94.1% 是测出来的,不是估出来的\n\nCompact 的行为保留率用重放法:取全精度 bf16 模型答对的 118 个条目,在量化版上重放,复现 111 个,即 94.1%。分类看:短事实 95.0%,grounded extraction 100%,false-premise 拒答 87.2%。官方同时给出测量门限约 ±3.4 个点——小于这个差距的差异当噪声处理。这种「连误差棒一起交」的写法,在模型发布里并不多见。\n\n## 配方:bit 预算花在哪\n\n- 120 个专家张量用 IQ2_XXS(2.0625 bpw),其余走 Q4_K_M 混合。专家占全部参数的 93%,贡献 1.92 bpw;剩余 0.38 来自 4\u002F6-bit 的非专家矩阵,加上 f32 的 router 和 norm\n- router 和所有归一化张量保持 f32——路由出错改变的是「这个 token 用到哪份知识」,是离散的错误,不能平滑退化;而 router 只有约 21M 参数,保护它几乎免费\n- imatrix 用 120 块、共 3MB 的语料算出:源码、技术文档和提问——刻意匹配这个模型的目标场景,而不是通用网页文本\n\n## 两条反直觉结论\n\n**一,range selection 支配一切。** 在同样字节预算下,把 min\u002Fmax 分组定范围换成 per-group MSE clip search,保留率从 77.1% 提到 95.8%——一个范围选择策略的贡献,比多数「更聪明的量化算法」都大。\n\n**二,保护输出头毫无收益。** 通行建议是重点保护 output head,但他们在这套配置下的仿真显示:head 和 embedding 放在 4-bit,118\u002F118 全保留。IQ2_XXS 加 imatrix 本身就在做重要性加权的范围搜索,这就是最终出货的 build。\n\n## 跑起来的坑\n\nCompact 可直接在 llama.cpp、Ollama、LM Studio 加载(需要 qwen3_5_moe 支持)。model card 明确列了几个「不是可选项」的 flag:--jinja 不加则工具调用解析失败、多轮工具使用直接挂;--reasoning-format deepseek 不加则推理内容堆进 content,每轮累积,模型重复执行直到预算耗尽;KV cache 建议 q8_0 而非 q4_0——权重已经压到 2.30 bpw,KV 再压会伤长程状态跟踪,表现为「重做已经做完的工作」。另外 MTP 层被禁用、视觉塔未包含,这是纯文本 build。\n\n## 顺带:Macaw 才是「给你们用的」\n\n配套发布的 Macaw 是跑在 Apple Silicon 上的本地 Mac 智能体:97 个验证过的工具覆盖邮件、文件、日历、音乐和系统设置,M2 上实测解码 40.3 tok\u002Fs、平均请求 1.21 秒、模型仅 1.5GB,支持一句话串起 3-5 个工具的多步链。基座是 LFM2.5,身份靠系统提示词注入;代码 MIT,权重继承 LFM Open License v1.0(商用上限:年营收 1000 万美元)。\n\n## 所以呢\n\n发布不到两周,Hugging Face 显示 BTL-4-Compact 近一月下载量已达 31,951。官方发布推文还给出 73.5% 的 BFCL v4(AST)成绩——厂商自报口径,待第三方验证。但这份 model card 真正的分量在于:2-bit 量化的每个决定都有数据支撑,连「保护输出头是白费」这种失败结论都写了出来。下一次有人抱怨「本地跑不动大模型」,先看 bit 数,再看参数量。\n\n参考:https:\u002F\u002Fhuggingface.co\u002Fbadtheorylabs\u002FBTL-4-Compact","https:\u002F\u002Fhuggingface.co\u002Fbadtheorylabs\u002FBTL-4-Compact","f3142574-d026-4167-b767-333da980de35",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ad43820e-dac1-4c2b-87ff-62d1bd2015b2","en","BTL-4: a 35B model squeezed into one 9.96GB file","Independent lab Bad Theory Labs open-sourced a 35B MoE agentic model BTL-4 on August 5, with a Compact build that packs the whole model into a single 9.96 GB file at 2.30 bits per weight, retaining a replay-measured 94.1% of behavior. The model card lays out the full recipe and two counterintuitive findings: range selection dominates everything else, and protecting the output head is worth nothing.","# 35B in a 9.96 GB file: inside BTL-4's 2.30-bit extreme quantization and two counterintuitive findings\n\nOn August 5, independent AI lab Bad Theory Labs released two open-weights models: BTL-4, a 35B agentic reasoning model, and Macaw, an on-device agent for the Mac. The BTL-4 Compact build takes one idea to its limit: **the entire 35B model in a single 9.96 GB file, 2.30 bits per weight on average, with a replay-measured 94.1% behavior retention**. The model card is worth a close read, because it lays out the complete engineering ledger of 2-bit-class quantization.\n\n## Architecture: a large model's memory, a small model's compute\n\nBTL-4 is a mixture of experts: 35.1B total parameters, roughly 2.1B active per token — the lab describes it as costing a large model's memory and a small model's compute. Of 40 layers, 30 are linear-attention and 10 full-attention; each layer has 256 experts with 8 routed per token. Native context is 262,144, and the KV cache only grows on 10 layers at ~20 KB\u002Ftoken — the whole 262K window costs about 5.2 GB of cache. Long-context work fitting on consumer hardware is the design goal, not an accident.\n\n## The 94.1% is measured, not estimated\n\nCompact's behavior retention uses replay: take 118 items the full-precision bf16 model answers correctly, replay them against this build, and it reproduces 111 — 94.1%. By category: 95.0% short-form factual, 100% grounded extraction, 87.2% false-premise rejection. The card also states the measurement gate at roughly ±3.4 points, so smaller differences should be treated as noise. Publishing the error bars along with the headline number is not something you see every day in model releases.\n\n## The recipe: where the bit budget goes\n\n- The 120 expert tensors use IQ2_XXS (2.0625 bpw); everything else follows the Q4_K_M mixture. Experts are 93% of all parameters and contribute 1.92 bpw; the remaining 0.38 comes from 4-bit and 6-bit non-expert matrices plus the f32 router and norms\n- The router and every normalization tensor stay at f32. A routing error changes *which* knowledge a token reaches — a discrete error that cannot degrade gracefully — and at ~21M parameters, protecting it is essentially free\n- The importance matrix was computed over 120 chunks of a 3 MB corpus of source code, technical documentation and question prompts — a deliberate match for what this model is for, rather than generic web text\n\n## Two counterintuitive findings\n\n**First, range selection dominates everything else at low bit widths.** At an identical byte budget, replacing min\u002Fmax group ranging with a per-group MSE clip search moved retention from 77.1% to 95.8% — the choice of range-selection strategy contributes more than most smarter quantization algorithms.\n\n**Second, protecting the output head is worth nothing.** The usual recommendation is to protect the output head, but simulation on this model showed head and embedding at 4-bit retained 118 of 118. IQ2_XXS with an imatrix already performs its own importance-weighted range search — which is why that is the build shipped here.\n\n## Running it: the flags that are not optional\n\nCompact loads directly in llama.cpp, Ollama and LM Studio (requires qwen3_5_moe support). The card lists several flags marked as not optional: without --jinja, llama.cpp ignores the embedded template and tool calls fail to parse, breaking multi-turn tool use; without --reasoning-format deepseek, reasoning stays in content and accumulates every turn until the model repeats itself out of budget; for the KV cache, q8_0 is recommended over q4_0 — the weights are already at 2.30 bpw, and compressing the cache on top of that degrades long-horizon state tracking, which shows up as the model redoing work it already completed. The MTP layer is disabled and the vision tower is not included: this is a text-only build.\n\n## Meanwhile, Macaw is the one for everyone\n\nThe companion release, Macaw, is a local Mac agent on Apple Silicon: 97 verified tools covering mail, files, calendar, music and system settings, measured at 40.3 tok\u002Fs decode on an M2, 1.21 s mean request, a 1.5 GB footprint, and multi-step chains that trigger 3-5 tools from one sentence. The base model is LFM2.5, with identity prompt-steered; the code is MIT and the weights inherit the LFM Open License v1.0, which caps commercial use at 10 million USD in revenue.\n\n## So what\n\nWithin two weeks of release, BTL-4-Compact shows 31,951 downloads in the last month on Hugging Face. The launch post also cites 73.5% on BFCL v4 (AST) — a vendor-reported number pending third-party verification. But the real weight of this model card is that every decision in 2-bit quantization comes with data attached, including the failure finding that protecting the output head bought nothing. The next time someone says a big model cannot run locally, check the bits per weight before the parameter count.\n\nReference: https:\u002F\u002Fhuggingface.co\u002Fbadtheorylabs\u002FBTL-4-Compact","btl-4-compact-9gb-2bit-quantization","2026-08-17T13:20:00Z","2026-08-17T13:11:48.690072Z","2026-08-17T13:11:48.690084Z",true,"agent",91,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"491f4904-c854-4925-b3e3-e34b8afd5e50","KDA+MLA 混合栈下沉到 1.3B 激活:Ling-3.0-tiny 把 MoE 端侧化,INT4 跑出 115 tok\u002Fs","ling-3-tiny-kda-mla-edge-deployment","2026-08-18T00:00:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"bb42bbdc-55a5-422c-ae68-4d103639bfd2","Ling-3.0-tiny：M4 Pro 实测 86-90 tokens\u002Fs","ling-3-0-tiny-edge-deployment-ledger","2026-08-17T23:30:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"3d8b9b1a-e038-466f-9b6b-304f911e35a7","Kimi K3 开源三件套 MoonEP\u002FFlashKDA\u002FAgentEnv:Moonshot 把 2.8T MoE 训练栈完整交底","kimi-k3-moonep-flashkda-agentenv","2026-07-28T04:30:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"031e715e-c9d6-4855-83da-0515f33f0e3c","POCKET：35B MoE 1-bit 跑进 iPhone，27 tok\u002Fs","pocket-35b-moe-iphone-edge","2026-07-28T04:00:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"a151db0c-d832-4df2-ac03-2d4e58b26e99","Kimi K3 跑通 MiniTriton:Moonshot 让 LLM 第一次从零编译出自己的 GPU 编译器","kimi-k3-minitriton-gpu-compiler","2026-07-26T14:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"93fb05a4-79cd-4abf-b263-c7d1910dbea7","Kog Laneformer 2B 开源:把推理引擎焊进 Transformer 架构,2B 模型单请求解码跑到 3000 tok\u002Fs","kog-laneformer-2b","2026-06-24T14:00:00+00:00"]