Running models on phones, watches and robots has mostly meant quantizing a general-purpose small model — a Gemma E2B-class base that tries to do a bit of everything. Cactus Compute's freshly open-sourced Needle 3 flips that: it deliberately gives up general chat and does exactly three jobs, packing a tool-calling foundation model into a single 8-29 MB file.

What it gives up on purpose

Needle 3 targets mobiles, wearables, robots, smart homes, cars and microcontrollers, but unlike general small models it does three things, all on-device:

  • Tool calls: given the functions an app exposes, it picks the right one and fills every argument. Two requests produce two calls in order; a request no tool covers returns an empty list, not a guess.
  • Structured extraction: declare a schema, hand over messy text, get typed fields back — invoices, bookings, notifications, forms. A byte-level grammar compiled from the schema constrains every token, so output is guaranteed to parse.
  • Text embedding: the same model returns a sentence vector, so an app can search, match and route locally.

The official claim: that capacity trade buys beating models 10x its size on mobile tool calls and matching 2-3x bigger models on extraction — vendor-reported, with benchmark charts on the model card (exact-match for tool calls, field micro-F1 for extraction).

One weight file, every depth from 2 to 20 layers

Architecturally, Needle 3 is a Laddered Simple Attention Network: the FFN is replaced by a Monarch Hadamard MLP, attention is GQA with causal conv taps, and most parameters live in an engram n-gram memory read by gather — the company says the 121M-parameter model does the arithmetic of a 50M one. Training is arranged so that every depth from 2 to 20 layers is a deployable model; you cut at build time to fit the device.

Weights are compressed with Cactus Quants to CQ 2-bit (2.125 bits per weight); the whole file is 8-29 MB and each platform's engine is under 1 MB. Another design worth noting: every response carries a calibrated confidence score from a learned head, which the engine uses to route between acting, confirming or refusing — a small model that knows when not to act.

Fine-tuning is where it gets serious

The heaviest numbers in the model card sit in the fine-tuning section: after fine-tuning on DroidCall, every subnetwork gains 18-36 points, and from 4 layers (29M parameters) up, the tuned subnetwork passes DeepSeek V4 Flash — again vendor-reported. The workflow is pragmatic: LoRA on the frozen 20-layer base, then one build command merges the adapter, slices any subnetwork and exports a 4-bit file that runs on the same engine.

Engineering maturity shows: Apache-2.0 license, 11.4k GitHub stars, 307 commits, and a generation=2 flag keeps Needle 2 deployments working. One wrinkle: telemetry is on by default in the binary, disabled via NEEDLE_TELEMETRY=0 — a small thorn in an on-device privacy story.

So what

Agent tool calling is splitting into layers: frontier models in the cloud, and what at the edge? Needle 3's answer is to swap the generalist small model's chat dreams for a dedicated base. Its bet: in an 8 MB world, filling arguments correctly is worth more than small talk. Whether that bet pays off shows up in who adopts it next year.

Refs: HF model card · GitHub repo