Talk to a voice assistant and it always lags half a beat — and the problem is usually not the LLM but the recognizer: streaming speech recognition waits until a word is finished before emitting it, typically 90–400 ms later. For a voice agent, that is time it could spend starting a search, prefetching records, or planning a reply. A new arXiv paper, Earlybird, flips that default: a 29M-parameter model with 4-bit weights of just 16.8 MB prints words before the speaker finishes them — on the audiobook test set the median word is output correctly 90 ms early — and attaches a calibrated confidence to every word.
Turning "wait" into "guess"
The recipe is straightforward. Earlybird takes Moonshine Tiny (34M, MIT license), cuts its vocabulary down to the 19,007 tokens English speech actually uses, shrinking it to 29M parameters with almost no accuracy loss (within 0.4 WER points on every test set), then runs quantization-aware training to 4 bits for a 16.8 MB weight file.
The core training trick is feeding randomly truncated audio: cut at random 80 ms steps, the model must write every word that is "due" at that moment — spelling half-heard words in full — plus a guess of the next two words. Because targets at successive cut points only ever extend one another, the model is never trained to take back what it has said. Two details matter: the distillation loss is applied only at token positions where the teacher is right (otherwise guessed words learn to be indecisive), and padding the encoder with three silence frames pushes the median earliness from 50 ms to 90 ms.
Six test sets, all early
The paper compares four released models (each run with its own official code in streaming mode) across six test sets with per-word timestamps:
| Test set | Earlybird | Moonshine Tiny | Whistle | Whisper base |
|---|---|---|---|---|
| Audiobooks clean | -90 ms / 2.3 | +270 / 3.9 | +400 / 3.1 | +90 / 2.3 |
| TED talks | -68 ms / 5.4 | +298 / 6.8 | +505 / 7.3 | +125 / 4.8 |
| Earnings calls | -46 ms / 18.4 | +309 / 21.6 | +535 / 17.8 | +142 / 13.3 |
It is the earliest on all six sets (the others: audiobooks other -60, public speech -66, synthetic -73). On audiobooks, 69% of words are predicted correctly before they end, against 3–25% for the comparison models. On the Open ASR Leaderboard's eight English sets, the 4-bit release averages 12.6 WER — beating the official Moonshine Tiny streaming runtime (14.4) on all eight, roughly tying same-size Whistle (12.2), while the full-precision model reaches 11.4, slightly better than the larger Whisper tiny.en (12.0). The compute story holds up too: 39 ms on average per 80 ms audio step on a single CPU thread, about twice real time. Most practical of all is confidence: for words still being spoken, expected calibration error is 3.7–5.5, and words output at confidence ≥0.9 are right 91–96% of the time — downstream agents get "word + trust score" and can set their own threshold.
The costs and the edges
Prediction is not free. Early output means more revisions: 1.29 take-backs per spoken word on audiobooks — far below the Moonshine runtime (2.07) but far above wait-it-out Whistle (0.16) — rising to 2–6 on spontaneous or noisy speech. Background chatter is the biggest weakness: at 10 dB, WER is 32.0 against 8.9 for Whisper base. The model is English-only, and calibration degrades on out-of-domain command speech (SLURP, ECE 8.8). And scaling up does not solve latency by itself: the purpose-built streaming Nemotron 0.6B (464 MB) lands at +230 ms median, while Parakeet TDT 0.6B — 70× Earlybird's size, re-decoded every 240 ms — reaches -10 ms but at half of Earlybird's error rate.
So what
What this paper really moves is not accuracy but the architectural default of waiting for words to end — a variable rarely treated as a design choice. Going from +270~+309 ms to -90 ms hands downstream agents nearly 400 ms of free thinking time, which in voice interaction may be worth more than another point of WER. With calibrated confidence, the recognizer hands the "trust it or not" decision to the caller for the first time. The weights are open-sourced under MIT on HuggingFace — 16.8 MB, twice real time on one CPU core — another piece of the on-device voice agent puzzle. Once the recognizer starts running ahead, is the next latency to optimize "before the speaker even opens their mouth"? (Paper: arXiv 2611.14421)