NVIDIA put Nemotron 3.5 ASR on Hugging Face on June 4: 600M parameters, one checkpoint, 40 language-region coverage; streaming output has built-in casing and punctuation, licensed under OpenMDW-1.1. This isn't another "Whisper successor" remake — it's an engineering sample of breaking the long-standing "streaming vs. accuracy" deadlock in speech AI.
The core is Cache-Aware FastConformer-RNNT. Traditional streaming ASR uses sliding windows to repeatedly recompute overlapping audio chunks, wasting half the compute on tokens already processed; Nemotron 3.5 instead caches the encoder's self-attention and convolution activations, computing each frame only once without repeated work. NVIDIA self-reports a 17× increase in concurrent streams on H100 over the baseline. The nicest part is the att_context_size parameterized knob: five latency tiers (80ms, 160ms, 320ms, 560ms, 1.12s) share the same weights, with the choice made at deployment time without retraining.
The most easily overlooked is the engineering meaning of "single checkpoint, many languages." The common practice is to deploy a separate model per language and stack a language-identification component on top; here the system uses a prompt-based target_lang conditional input for language routing, and target_lang=auto can even output a language tag after terminal punctuation. Customer service, meeting transcription, multilingual mixed podcasts no longer need a "museum" of 40 models.
Compared with commercial APIs: Whisper large-v3 runs offline batch processing, Deepgram Nova-3, AssemblyAI Universal-3 Pro, and ElevenLabs Scribe v2 Realtime are all closed-source streaming models that don't support local fine-tuning. Nemotron 3.5's open weights mean continued training is possible for small languages like Greek and Bulgarian — the samples NVIDIA provides show Greek FLEURS WER dropping from 35 to 24 (32% relative), and Bulgarian from 22 to 15 (31% relative), both under the most aggressive 80ms mode.
On the eve of the voice-agent explosion, throughput and time-to-first-token are the production-grade dividing line, and Nemotron 3.5 at least stuffs "self-hosted, low-latency, fine-tunable" into a 600M model for the first time. The real test is when gRPC streaming interfaces on NIM ship — the last mile between open source and commercial deployment often decides whether a model can actually go into production.