Liquid AI released LFM2.5-8B-A1B, an 8B-parameter MoE with only 1.5B active per token. The standout: the model matches the quality of much larger dense models, at the inference cost of a 1.5B model. The "edge LLM" ceiling is pushed up significantly.
The "8B / 1.5B active" architecture: LFM2.5-8B-A1B has 8B total parameters, with 1.5B active per token. The "extreme sparsity" (18.75% active) is achieved through a careful expert design — each expert is a "specialist" in a specific domain, and the router dispatches to the right expert with high accuracy. The "1.5B active" is small enough to run on a MacBook M2 at 80 tokens/sec.
The benchmark: on MMLU, HumanEval, and GSM8k, LFM2.5-8B-A1B scores within 2-3 points of Qwen2.5-7B (a dense 7B model), at 5× the inference speed. The "small active" + "large total" combination is the right architecture for edge LLMs.
The "edge LLM ceiling" highlight: the previous "edge LLM" ceiling was around 3-4B dense (e.g., Phi-3-mini, Gemma-2-2B). LFM2.5-8B-A1B pushes this ceiling to 8B (with 1.5B active), opening up new use cases for on-device LLMs. The model can handle complex tasks (code generation, multi-step reasoning) that were previously only possible with cloud LLMs.
The bigger takeaway: "extreme sparsity" is the right architecture for edge LLMs. The "dense is best" assumption is breaking, and the "MoE with small active + large total" approach is significantly more efficient. For the industry, this signals that "edge LLMs" will move to extreme-sparsity MoE, and the next round of edge AI products will be defined by "how sparse the MoE is."