A Model Card That Says "No Datacenter Required"

inclusionAI has open-sourced Ling-3.0-tiny, the entry-level member of the Ling 3.0 family: a hybrid-reasoning MoE model with 7.9B total parameters and only 1.3B activated per token, released under the MIT license with weights on Hugging Face. Unlike the production-oriented Flash tier of the same family, tiny has one clear goal — bringing hybrid reasoning to local and resource-constrained deployments.

Architecture: 3:1 KDA+MLA Stacking, Plus a 128-Pick-9 Sparse MoE

Ling-3.0-tiny inherits the hybrid linear-attention architecture of the Ling-3.0 series, built on two stacked ideas:

  • 3:1 KDA-MLA stacking: every 4-layer block combines 3 layers of Kimi Delta Attention (KDA) with 1 layer of Multi-Head Latent Attention (MLA). The team says this balances long-context processing efficiency, parameter efficiency, and computational cost;
  • Sparse MoE FFN: out of 128 routed experts, each token activates only 8 routed experts plus 1 shared expert — which is exactly why a 7.9B-parameter model only moves 1.3B per token.

The model natively supports hybrid reasoning: thinking mode is on by default and can be turned off per request via enable_thinking, so a single model serves both fast responses and multi-step reasoning. BF16, FP8, and INT4 weights are all provided, covering deployment options from servers to consumer hardware.

The Edge Ledger: Three Sets of Numbers

The most interesting part of this model card is that it directly publishes measured numbers on consumer hardware:

Hardware Precision Speed
NVIDIA DGX Spark FP8 ~100-105 tokens/s
M4 Pro MacBook FP8 ~86-90 tokens/s

At an 8K context length, peak memory sits around 8.34 GiB. In Artificial Analysis testing, output speed exceeds 160 tokens/s, with roughly 18 seconds of end-to-end latency for a 500-token response including reasoning time; it scores 25 on the AA Intelligence Index v4.1.1 and 16 on the AA Agentic Index.

Toolchain Reality: One Model, Three Maturity Levels

The deployment paths deserve their own section — this may be the most honest cross-section of the edge-model ecosystem today:

  • SGLang: smoothest. The official prebuilt image lmsysorg/sglang:dev-Ling-3.0-tiny pulls and runs; the low-latency recipe ships with MTP/NEXTN speculative decoding built in, YaRN-extends context to 256K, and a single 141GB-class GPU (H20-3e) is enough to serve it;
  • vLLM: requires a community branch (the ling_3_0 branch, compiled yourself) — not yet in mainline;
  • Ollama: support is still at the PR stage (#17643), limited to the MLX path on Apple Silicon, self-compiled, and not yet part of an official release — even though the team validated it on an M4 Pro with 48GB of unified memory.

Commentary: The Signal Beyond the Numbers

Judging purely by its AA Intelligence Index score of 25, tiny is not spectacular — but its selling point was never leaderboard rank. The combination of "1.3B activated parameters + 8.34 GiB memory" makes hybrid reasoning run at usable speeds on a MacBook. Community uptake is real: a bit over a week after release, 13 community quantizations already exist on HF.

The toolchain gradient is the more telling signal: the same model gets an official SGLang image, a vLLM branch awaiting merge, and an Ollama PR still in flight. For edge open-weight models, a "release" stopped being just dropping a bag of weights — how deeply inference frameworks adapt determines how many people can actually run it. The MIT license removes commercial-use concerns as well.

For developers who want to run agentic workloads locally, this model card is a ledger you can hold against a procurement budget. For the industry, it confirms a trend: the hybrid linear-attention + sparse MoE combo is moving down from flagship exclusivity into the 8B class.

Source: inclusionAI Ling-3.0-tiny model card (Hugging Face): https://huggingface.co/inclusionAI/Ling-3.0-tiny