35B in a 9.96 GB file: inside BTL-4's 2.30-bit extreme quantization and two counterintuitive findings

On August 5, independent AI lab Bad Theory Labs released two open-weights models: BTL-4, a 35B agentic reasoning model, and Macaw, an on-device agent for the Mac. The BTL-4 Compact build takes one idea to its limit: the entire 35B model in a single 9.96 GB file, 2.30 bits per weight on average, with a replay-measured 94.1% behavior retention. The model card is worth a close read, because it lays out the complete engineering ledger of 2-bit-class quantization.

Architecture: a large model's memory, a small model's compute

BTL-4 is a mixture of experts: 35.1B total parameters, roughly 2.1B active per token — the lab describes it as costing a large model's memory and a small model's compute. Of 40 layers, 30 are linear-attention and 10 full-attention; each layer has 256 experts with 8 routed per token. Native context is 262,144, and the KV cache only grows on 10 layers at ~20 KB/token — the whole 262K window costs about 5.2 GB of cache. Long-context work fitting on consumer hardware is the design goal, not an accident.

The 94.1% is measured, not estimated

Compact's behavior retention uses replay: take 118 items the full-precision bf16 model answers correctly, replay them against this build, and it reproduces 111 — 94.1%. By category: 95.0% short-form factual, 100% grounded extraction, 87.2% false-premise rejection. The card also states the measurement gate at roughly ±3.4 points, so smaller differences should be treated as noise. Publishing the error bars along with the headline number is not something you see every day in model releases.

The recipe: where the bit budget goes

  • The 120 expert tensors use IQ2_XXS (2.0625 bpw); everything else follows the Q4_K_M mixture. Experts are 93% of all parameters and contribute 1.92 bpw; the remaining 0.38 comes from 4-bit and 6-bit non-expert matrices plus the f32 router and norms
  • The router and every normalization tensor stay at f32. A routing error changes which knowledge a token reaches — a discrete error that cannot degrade gracefully — and at ~21M parameters, protecting it is essentially free
  • The importance matrix was computed over 120 chunks of a 3 MB corpus of source code, technical documentation and question prompts — a deliberate match for what this model is for, rather than generic web text

Two counterintuitive findings

First, range selection dominates everything else at low bit widths. At an identical byte budget, replacing min/max group ranging with a per-group MSE clip search moved retention from 77.1% to 95.8% — the choice of range-selection strategy contributes more than most smarter quantization algorithms.

Second, protecting the output head is worth nothing. The usual recommendation is to protect the output head, but simulation on this model showed head and embedding at 4-bit retained 118 of 118. IQ2_XXS with an imatrix already performs its own importance-weighted range search — which is why that is the build shipped here.

Running it: the flags that are not optional

Compact loads directly in llama.cpp, Ollama and LM Studio (requires qwen3_5_moe support). The card lists several flags marked as not optional: without --jinja, llama.cpp ignores the embedded template and tool calls fail to parse, breaking multi-turn tool use; without --reasoning-format deepseek, reasoning stays in content and accumulates every turn until the model repeats itself out of budget; for the KV cache, q8_0 is recommended over q4_0 — the weights are already at 2.30 bpw, and compressing the cache on top of that degrades long-horizon state tracking, which shows up as the model redoing work it already completed. The MTP layer is disabled and the vision tower is not included: this is a text-only build.

Meanwhile, Macaw is the one for everyone

The companion release, Macaw, is a local Mac agent on Apple Silicon: 97 verified tools covering mail, files, calendar, music and system settings, measured at 40.3 tok/s decode on an M2, 1.21 s mean request, a 1.5 GB footprint, and multi-step chains that trigger 3-5 tools from one sentence. The base model is LFM2.5, with identity prompt-steered; the code is MIT and the weights inherit the LFM Open License v1.0, which caps commercial use at 10 million USD in revenue.

So what

Within two weeks of release, BTL-4-Compact shows 31,951 downloads in the last month on Hugging Face. The launch post also cites 73.5% on BFCL v4 (AST) — a vendor-reported number pending third-party verification. But the real weight of this model card is that every decision in 2-bit quantization comes with data attached, including the failure finding that protecting the output head bought nothing. The next time someone says a big model cannot run locally, check the bits per weight before the parameter count.

Reference: https://huggingface.co/badtheorylabs/BTL-4-Compact