Inkling-Small Is Not Just a Compressed Model: It Reworks the Multimodal MoE Efficiency Trade-Off

Thinking Machines Lab released the open-weight Inkling-Small model on July 30. Despite the word “Small,” it still contains 276 billion total parameters, with 12 billion activated for each token. The earlier Inkling model has 975 billion total parameters and 41 billion active parameters, making the new model roughly one quarter of its size by both measures. Inkling-Small is licensed under Apache 2.0, its weights are available on Hugging Face, and it can also be tested and fine-tuned through Tinker. The central claim in the official release is not merely that the model is smaller, but that it preserves or improves reasoning and agentic performance while using substantially less compute.

How the model reduces its active footprint

Inkling-Small is a 42-layer, decoder-only Transformer with a sparse Mixture-of-Experts feed-forward backbone. For every token, the router selects 6 of 256 experts, while 2 shared experts remain active for all tokens. The model accepts text, image, and audio inputs natively. Each modality is projected into a shared hidden space and processed jointly by the same decoder. It supports a context window of up to one million tokens and produces text output.

The smaller design is not simply a mechanical compression of the older model. Thinking Machines Lab says Inkling-Small began training after Inkling, which allowed the team to revise both the pre-training data mixture and the machine-learning recipe. An earlier preview checkpoint was post-trained partly through on-policy distillation, with Inkling acting as the teacher. The team then continued scaling agentic coding reinforcement learning for two additional weeks. In other words, the company redesigned the training path around a lower active-parameter budget rather than merely shrinking the original checkpoint.

Better coding and reasoning, but a sharp factuality regression

The official evaluation suite reports 80.2% on SWE-bench Verified for Inkling-Small, compared with 77.6% for Inkling. On Terminal-Bench 2.1, the scores are 64.7% and 63.8%, respectively. On the text-only version of Humanity’s Last Exam, Inkling-Small reaches 31.6%, ahead of Inkling at 29.7%. These results support the lab’s conclusion that the smaller model can outperform its teacher on reasoning, coding, and tool-use tasks.

The trade-off is equally visible. On SimpleQA Verified, Inkling-Small drops to 20.6% from Inkling’s 43.9%. On Tau 3 Banking, it scores 15.5%, compared with 23.7% for the larger model. Thinking Machines Lab explicitly states that Inkling retains an advantage in knowledge coverage and factuality. That contrast matters more than a single headline benchmark: reinforcement learning and distillation can concentrate limited compute on completing tasks, but they do not automatically restore broad factual knowledge.

The deployment threshold is lower, not low

Inkling-Small is distributed in BF16 and NVFP4 formats, among others listed in the model card. The BF16 checkpoint requires at least 600 GB of aggregated VRAM, which can be provided by four NVIDIA B300 GPUs or eight H200 GPUs. The NVFP4 checkpoint lowers the requirement to at least 180 GB. It can run in W4A4 mode on one B300, subject to the SM100-or-newer requirement, or in W4A16 mode on two H200 GPUs. Supported deployment frameworks include SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face.

This is therefore not a “small model” that runs on an ordinary consumer GPU. Its practical significance is that a native text-image-audio model moves from a very large cluster requirement to a single data-center GPU or a pair of H200s. For teams that need private deployment, tool use, and multimodal input, that reduction is more meaningful than the label attached to the parameter count.

Inkling-Small points to a broader direction for open-weight models. The next stage of competition is not only about increasing total parameters. It is about jointly optimizing activated parameters, reasoning effort, quantization formats, and training recipes. A useful small model is not one with a smaller name; it is one that completes more work per unit of compute while clearly handing its factual weaknesses to retrieval and verification systems.