Why Q4_0 is the edge-deployment hard case

To run a chat-capable LLM on a 16GB MacBook or a Raspberry Pi, weights have to drop below 4 bits. In the GGUF ecosystem the most common 4-bit formats are Q4_0 and Q4_K_M: the former is the fastest but the roughest, the latter slightly better at slight throughput cost. Q5_K_M typically adds another 1-2 accuracy points over Q4_K_M, at the cost of 25% more file size. That means any accuracy recovery done on top of Q4_0 effectively gives you something faster than Q4_K_M with Q5_K_M-like accuracy for free.

The traditional approach is post-training quantization (PTQ), where you round trained BF16 weights directly to 4 bits. The accuracy hit is particularly severe on small models like LFM2.5-230M, where weight headroom is thin to begin with. PTQ's Q4_0 GGUF averages only about 80-90% of BF16 on the six-to-seven task suite covering GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4, and the relative loss is clearly worse at 230M and 350M than at 2.6B.

Liquid AI's fix: teach the student to accept 4-bit during training

In the August 19 release, Liquid AI retrained the Q4_0 variants of all four sizes (230M / 350M / 1.2B-Instruct / 2.6B) with Quantization-Aware Distillation (QAD). The core idea: a high-precision teacher model trains a "quantized-aware" student during the training loop — the student is not rounded after the fact but learns end-to-end through simulated 4-bit gradient backpropagation. The released GGUFs share the same file format, tensor layout, and llama.cpp path as native Q4_0, so there is no throughput penalty: QAD Q4_0 runs at native Q4_0 speed while matching Q5_K_M accuracy.

What the numbers actually look like

Liquid AI ran the full benchmark suite on four real hardware backends:

  • MacBook Pro (Apple M5 Max): GPU inference, 100 generated tokens after a 256-token prompt prefill.
  • NucBox EVO-X2 (AMD Ryzen AI Max+ 395): same protocol.
  • Samsung Galaxy S26 Ultra (Qualcomm Snapdragon 8 Elite Gen 5): Arm CPU inference.
  • Raspberry Pi 5 (Broadcom BCM2712): Arm CPU inference; small models use a 32-token decode-only throughput run.

Recovery is defined as the percentage of the BF16→PTQ Q4_0 quality gap that QAD closes. Across the four sizes the results are 70.6% (230M) / 73.4% (350M) / 65.5% (1.2B) / 48.4% (2.6B); translated to "QAD Q4_0 retains how much of BF16", that is 97.1% / 96.5% / 97.4% / 96.6%. Even on 230M — the model PTQ hit hardest — QAD lifts Q4_0's accuracy to within run-to-run variance of Q5_K_M. On 1.2B it trails Q5_K_M by less than 0.5 points.

On 1.2B and 2.6B, QAD Q4_0 runs 3-14% faster than standard Q4_K_M at parity accuracy. On 230M and 350M, it runs 4-33% faster than Q5_K_M at parity accuracy. It also ties Unsloth's UD-Q4_K_XL, which the community has been treating as the 4-bit SOTA external PTQ checkpoint.

Why this is worth looking at

QAD is not a new idea, but industry releasing it as an engineering output (rather than a paper concept) and running the full benchmark suite against PTQ, Q5_K_M, Q4_K_M, and UD-Q4_K_XL in the same controlled comparison — rather than just "ours vs ours" — gives this release high marginal information. It overturns the long-standing assumption that Q4_0 means "cheap but not deployable".

The practical edge implication: on a 16GB MacBook, a Ryzen AI Max+ NUC, a Snapdragon 8 Elite phone, and a Raspberry Pi, Q4_0 now runs at Q5_K_M accuracy while being smaller and faster. The GGUFs are public on Hugging Face, with filenames suffixed QAD-Q4_0 and consumed by llama.cpp out of the box. In LLM inference, file size and throughput usually decide shippability more than a benchmark number — and this release lands exactly on that axis.

Sources: Liquid AI's official blog post "LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment" (https://www.liquid.ai/blog/qad), and the Hugging Face team blog "LFM2.5 Q4_0 Checkpoints from Quantization-Aware Distillation" (https://huggingface.co/blog/LiquidAI/qad). Both went live on August 19, 2026, with identical benchmark suites and hardware configurations.