Speech tooling has always been fragmented: one model for synthesis, another for denoising, another for separation, and pitch or emotion edits require yet more tools. On September 9, Tencent's Hunyuan team open-sourced AuK, a 1.5B-parameter speech foundation model that pulls generation and editing into a single interface built on natural-language instructions plus audio context.

Five Task Families, One Instruction Interface

According to the technical report, AuK was trained across five task families — speech generation, content editing, enhancement and separation, paralinguistic editing, and acoustic editing — mapping to 16 concrete tasks: zero-shot TTS, reference-free instruct TTS, speech content editing, melody-preserving lyric editing, semitone-level pitch control, speed and volume adjustment, emotion and timbre changes, de-accenting, adding or removing nonverbal sounds like breaths and laughs, whisper conversion, speech enhancement, speech separation, music separation, and content-based target speaker extraction.

The data scale behind this is substantial: roughly 3.03 billion instruction-audio instances and 1.95 million hours of effective supervision. Every task shares the same message-based call — one instruction string, optional audio input.

Architecture: MLLM + Joint VAE + Hybrid Rectified Flow

AuK combines three components: a multimodal LLM for semantic conditioning (the repo discloses the current implementation uses Qwen2.5-Omni-3B as encoder), a VAE jointly trained on speech, general audio, and music for acoustic conditioning, and a hybrid rectified-flow Transformer that runs dual-stream MMDiT blocks followed by unified single-stream DiT blocks.

Training proceeds in stages: generation-only warm-up, then joint generation-editing pre-training, then diverging post-training — human-feedback preference optimization for open-ended editing and reward-based reinforcement learning for speech generation.

AuK-Flash: 4-Step Inference, 4.5x Faster

Inference cost is compressed via distillation: consistency initialization plus task-routed Decoupled DMD yields AuK-Flash, which performs fixed 4-step inference without classifier-free guidance and achieves a 4.5x wall-clock speedup over the full model under matched conditions. The report claims leading performance on zero-shot and instruction-controlled speech generation and general instruction-guided editing, while remaining competitive on signal-level restoration — an official-eval caveat pending independent replication.

Open Delivery: MIT License and a Full Toolchain

The release is unusually complete: code and weights for both AuK and AuK-Flash under the MIT license, weight downloads and live demos on HuggingFace and ModelScope, ComfyUI nodes (ComfyUI-AuK), CLI and Python APIs, and a fine-tuning pipeline built on JSONL data with dynamic batching. The repo picked up 73 stars on release day.

Paper and code: arXiv:2609.08936 · GitHub repo

So What

AuK's approach is notable: rather than chasing benchmarks on a single speech task, it ports the instruction-unified, large-scale multi-task pretraining playbook — proven in LLMs — to the speech modality, mirroring this year's unified-model trend in image and video. At 1.5B parameters with a 4-step distilled variant, the route is already brushing against local-deployment thresholds.

For speech-application developers, MIT licensing plus a ready fine-tuning pipeline removes nearly all friction to trying it. For vendors of vertical speech tools, the question is sharper: when generation, editing, and separation collapse into one 1.5B model, how wide is the moat around any single-point feature?