Boson AI released Higgs Audio v3, a 4B-parameter text-to-speech (TTS) model that supports over 100 languages with native controllability. The standout: the model can be controlled via natural language instructions ("speak slowly," "in a happy tone," "with a British accent"), with the control integrated into the speech generation process.
The "natively controllable TTS" highlight: most TTS models offer limited control — you can pick from a set of pre-defined voices, but you can't control the speaking style in real time. Higgs Audio v3 takes natural language instructions as input, and the model generates speech that matches the instructions. The control is "native" — the model is trained end-to-end to follow instructions, not via a separate "style control" module.
The technical details: Higgs Audio v3 uses a "fusion" architecture that combines (1) a text encoder (for the input text); (2) an instruction encoder (for the control instruction); (3) a speech decoder (for the output audio). The three are trained jointly, with the instruction encoder providing "soft conditioning" to the speech decoder.
The "100+ languages" coverage: the model is trained on 100+ languages, including low-resource languages (Swahili, Bengali, Tagalog, etc.). The quality on low-resource languages is significantly better than previous TTS models, which often had "robotic" pronunciation for non-English languages.
The benchmark: on the TTS quality benchmark, Higgs Audio v3-4B scores within 0.8 points of the best closed-source TTS models. The "controllability" benchmark shows that the model follows 92% of natural language instructions correctly.
The bigger takeaway: "natively controllable TTS" is a significant new direction. The "predefined voices" assumption is breaking, and the "natural language control" approach is significantly more flexible. For the industry, this signals that TTS will move to "natural language controllable" models, and the next round of TTS products will be defined by "how expressive the control is."