CosyVoice3 is one of the stronger open-source TTS models, but its inference stack is built on the Python ecosystem: PyTorch plus a long tail of version-pinned dependencies that adds up to several gigabytes and forces every deployer to play package detective. Velum, open-sourced by HardenedLinux on GitHub, rewrites the entire text-to-speech pipeline in modern C++ and ships a single executable that needs no Python at runtime.
Engineering trade-off: zero Python online
The neural networks (LLM / Flow / HiFT) run on GGML with CUDA acceleration; weight conversion and acoustic frontend steps (campplus, the speech tokenizer, and the matcha 80-bin mel) are pushed into an offline Python step that runs once and produces frozen files the C++ binary loads verbatim. The repo's CMake project defaults to VELUM_ENABLE_CUDA=ON and falls back to a CPU-only build when no CUDA toolchain is detected; both backends are linked into the same binary, the runtime auto-selects, and VELUM_BACKEND=cpu forces the CPU path.
Stage-by-stage numerical alignment with PyTorch
Each cross-check reports a scale-normalized relative error: <=1e-2 is GREEN (passes), 1e-2 to 1e-1 is YELLOW (needs investigation), >1e-1 is RED (structural divergence, hard fail). The repo ships four verify_*.py scripts covering the DSP frontend (Whisper 128-bin log-mel plus Kaldi 80-bin fbank), the Flow decoder (PreLookaheadLayer + DiT x22 + CFM Euler), the HiFT vocoder, and the LLM backbone (Qwen2-0.5B + CosyVoice3LM heads + ras_sampling). The end-to-end CLI test, tests/verify_e2e.py, runs the full LLM -> Flow -> HiFT chain on seed 0 against PyTorch. On the CPU backend, Flow mel max abs is 1.142e-3 with relative error 1.056e-4, HiFT pcm relative error 8.112e-3; on CUDA the numbers are 2.220e-3 / 2.052e-4 and 3.495e-3 / 5.036e-3 respectively. Both backends sit inside the GREEN band.
Who wrote it, and what is still Python
This is also an unusual example of LLM-authored C++. The README credits deepseek-v4-pro for the LLM work and Claude Code as the coding assistant, with human-driven architecture. Caveats: the ONNX frontend (campplus + speech tokenizer) and the matcha 80-bin mel are still "temporarily handed to Python" and produce files the CLI consumes rather than running them in C++; the LLM weight file llm.gguf is around 2.6 GB and the CUDA path needs at least that much free VRAM, so the docs recommend forcing the CPU backend if cudaMalloc reports out of memory.
For teams trying to embed TTS into products or agent pipelines, Velum's value is the deployment simplification: one binary, no Python runtime, numerical alignment with the PyTorch reference at every stage, and CPU/CUDA selectable from the same executable. The repository is GPL-3.0 licensed.