While most general audio models keep iterating on the diffusion Transformer paradigm, StepFun's StepFun-Audio Team has released a technical report with 71 authors. The model inside it packs zero-shot TTS, voice design, vocal generation, sound effects, music, and full multi-element soundscapes into a single framework — and does so by going the opposite way, back to discrete autoregression.

Going against the grain: RVQ tokens, not diffusion

The arXiv abstract is blunt: this is a discrete autoregressive generator that models audio directly over residual vector quantization (RVQ) tokens, departing from the diffusion Transformer-based continuous generation paradigm prevalent in recent general audio models. Three pieces make it work:

  • Tokenizer. StepAudio Tokenizer represents general audio at 12.5 Hz in a shared 16×2048 residual code space, jointly quantizing semantic and waveform-level acoustic features so every code layer preserves both types of information.
  • Split backbone. The backbone predicts the first codebook autoregressively along the time axis; a lightweight causal Transformer completes the remaining fifteen codebooks along the codebook axis.
  • Shared representation. Speech, vocals, sound effects and music all land on the same discrete representation — the precondition for one model doing every audio job.

Three design principles: don't break the LLM

The team distilled three lessons. First, interference-aware progressive pretraining, which acquires audio capabilities while preserving the textual abilities of the underlying large language model. Second, the RVQ Adaptor for effectively incorporating multi-codebook acoustic representations. Third, discrete autoregressive modeling over a shared representation across general audio domains. Training runs progressive pretraining, then multi-task instruction training, then supervised fine-tuning. The first principle deserves attention: audio capability crowding out text capability is the classic failure mode when an LLM changes careers into audio.

Official arenas: first on both boards — read the fine print

The project page ships two arena charts. On the Chinese Human-Likeness Arena, the model scores Elo 1755.3, ranked first among the six models shown, 211.1 points ahead of the next one; across 500 head-to-head trials it posted 410 wins, 39 ties and 51 losses — an 82.0% win rate — with per-opponent win rates of 73% vs Qwen, 78% vs the previous-generation StepAudio 2.5 TTS, 79% vs Doubao, and 90% vs both Inworld and MiniMax. On the Voice Design Arena it scores Elo 1668.5, first among the five models shown, 97.0 points ahead, with 148 wins in 196 trials for a 75.5% overall win rate.

Two caveats. The arenas and the comparison lists both come from the official project page, so the paper's claim of state-of-the-art performance on TTS and voice design is self-reported, with no independent replication yet. And nothing shipped with the report: the Hugging Face paper page lists zero linked models, and the project page only shows "Voice Studio Coming Soon".

So what

The story isn't "another audio model" — it's the route choice: at a moment when diffusion feels like the default answer, this team bet on discrete autoregression plus a shared code space and backed it with arena numbers. Full-element generation aims straight at film, radio-drama and short-video production: dialogue, ambience, effects and music arranged on a single timeline, skipping cross-craft stitching. The question worth tracking next: will this 12.5 Hz shared code space become infrastructure for audio generation, the way text tokenizers did for language?

References: arXiv:2609.12945 (https://arxiv.org/abs/2609.12945); project page stepaudiollm.github.io/step-audio-3-gen; Hugging Face paper page, 28 upvotes.