PrismML, a Caltech-affiliated team, has released Bonsai 8B, claiming it is the first 1-bit large language model with commercial viability. The core breakthrough: an 8B-parameter language model, after 1-bit quantization, has weights that occupy just 1.15GB of memory — roughly 1/12 to 1/14 of an equivalent FP16 model.
The 1-bit quantization principle is straightforward: each weight stores only one bit (0 or 1), with 0 mapped to -scale and 1 mapped to +scale. Every 128 weights share one FP16 scaling factor, preserving the model's distribution characteristics at extreme precision. The biggest benefit of this extreme compression is the dramatic drop in deployment cost — 1.15GB means the model can run directly on phones, embedded devices, or even in a browser, without a GPU or cloud API.
In benchmarks, Bonsai 8B maintains competitive performance among 8B-class models. While 1-bit quantization inevitably brings precision loss, PrismML's improved post-training quantization algorithm and scaling-factor optimization keep the model close to full-precision peers on reasoning, common-sense, and coding tasks. This shatters the old notion that 1-bit quantization is only good for demos.
PrismML emerged from stealth on March 31; Bonsai 8B's open-source weights are already available in MLX format. For edge AI and on-device inference scenarios, this is a direction worth watching: when models are small enough that deployment cost is negligible, the form factor of AI applications will fundamentally change.