[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-stepaudio-3-gen-rvq-autoregressive":3,"topics-all":35,"news-related-a18ac6b0-2c9b-4172-87bd-0efe079edc7d":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"a18ac6b0-2c9b-4172-87bd-0efe079edc7d","StepAudio 3 Gen：一个模型生成整个声场，官方竞技场两榜居首","阶跃星辰 StepFun-Audio Team 发布 StepAudio 3 Gen 技术报告：以 12.5Hz、16×2048 共享残差码空间的离散自回归架构统一 TTS、音色设计、人声、音效与音乐；官方竞技场两榜居首，但评分与对比名单均出自官方，权重未随报告放出。","当大多数通用音频模型还在 diffusion Transformer 的连续生成范式里迭代时，阶跃星辰 StepFun-Audio Team 交出了一份 71 位作者署名的技术报告。报告里的模型把零样本 TTS、音色设计、人声演唱、音效、音乐，以及多类音频混合的\"全要素声场\"，全部塞进同一个框架——而且反着来，回到离散自回归。\n\n## 反着来：不做 diffusion，在 RVQ token 上自回归\n\narXiv 摘要写得很直接：这是一个离散自回归生成器，直接在残差向量量化（RVQ）token 上建模音频，脱离近期通用音频模型流行的 diffusion Transformer 连续生成范式。拆开看是三件事：\n\n- **Tokenizer**：StepAudio Tokenizer 以 12.5 Hz 表示通用音频，使用 16×2048 的共享残差码空间，语义特征与波形级声学特征联合量化，每层码本都同时保留两类信息；\n- **骨干分工**：主干沿时间轴自回归预测第一本码书，剩余 15 本码书由一个轻量因果 Transformer 沿码本轴补完——时间轴和码本轴各管各的；\n- **共享表示**：语音、人声、音效、音乐全部落在同一套离散表示上，这是\"一个模型干所有音频活\"的前提。\n\n## 三个设计原则，核心是别把 LLM 教坏\n\n团队总结了三条经验：一是干扰感知的渐进式预训练，在获得音频能力的同时保住大语言模型原有的文本能力；二是 RVQ Adaptor，把多码本声学表示有效接进主干；三是跨音频域共享表示上的离散自回归建模。训练路径是渐进预训练、多任务指令训练加监督微调三段式。第一条尤其值得注意——音频能力吃掉文本能力，是 LLM 改行做音频最常见的翻车点。\n\n## 官方竞技场：两榜居首，但要看清口径\n\n官方项目页放出了两张竞技场图表。中文拟人度竞技场上，该模型 Elo 1755.3，在图中展示的六个模型里排第一，领先次名 211.1 分；500 场对抗 410 胜 39 平 51 负，总胜率 82.0%——对 Qwen 73%、对上代 StepAudio 2.5 TTS 78%、对 Doubao 79%、对 Inworld 和 MiniMax 均为 90%。音色设计竞技场上 Elo 1668.5，五个模型中居首，领先 97.0 分，196 场 148 胜，总胜率 75.5%。\n\n两点口径必须说清：竞技场与对比名单都来自官方项目页，论文\"TTS 与音色设计达到 SOTA\"属于团队自报，尚无独立第三方复现；同时报告中没有给出权重或代码获取入口，Hugging Face 论文页关联模型数为 0，项目页只挂着\"Voice Studio Coming Soon\"。\n\n## 所以呢\n\n看点不在\"又多一个音频模型\"，而在路线选择：在 diffusion 几乎成为默认答案的时段，这支团队押注离散自回归加共享码空间，并拿竞技场数据自证。全要素生成直接对准影视、广播剧、短视频的生产流——角色对白、环境、音效、音乐在同一条时间线上编排，省掉跨工种后期拼接。下一个值得盯的问题是：这套 12.5 Hz 共享码空间，会不会像文本 tokenizer 一样，成为音频生成的基础设施？\n\n参考：arXiv:2609.12945（https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.12945）；项目页 stepaudiollm.github.io\u002Fstep-audio-3-gen；Hugging Face 论文页 28 upvotes。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.12945","5f7d17cd-f95b-4a76-be2e-db79144de285",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b1853a5a-d940-42b7-94f9-0488ee3f2cf7","new-model",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"5a271684-9974-416b-9536-5e6d36b4f0a1","en","StepAudio 3 Gen: RVQ autoregression for the whole soundscape","StepFun's StepAudio 3 Gen unifies TTS, voice design, vocals, SFX and music in one RVQ autoregressive model, first in official arenas.","While most general audio models keep iterating on the diffusion Transformer paradigm, StepFun's StepFun-Audio Team has released a technical report with 71 authors. The model inside it packs zero-shot TTS, voice design, vocal generation, sound effects, music, and full multi-element soundscapes into a single framework — and does so by going the opposite way, back to discrete autoregression.\n\n## Going against the grain: RVQ tokens, not diffusion\n\nThe arXiv abstract is blunt: this is a discrete autoregressive generator that models audio directly over residual vector quantization (RVQ) tokens, departing from the diffusion Transformer-based continuous generation paradigm prevalent in recent general audio models. Three pieces make it work:\n\n- **Tokenizer.** StepAudio Tokenizer represents general audio at 12.5 Hz in a shared 16×2048 residual code space, jointly quantizing semantic and waveform-level acoustic features so every code layer preserves both types of information.\n- **Split backbone.** The backbone predicts the first codebook autoregressively along the time axis; a lightweight causal Transformer completes the remaining fifteen codebooks along the codebook axis.\n- **Shared representation.** Speech, vocals, sound effects and music all land on the same discrete representation — the precondition for one model doing every audio job.\n\n## Three design principles: don't break the LLM\n\nThe team distilled three lessons. First, interference-aware progressive pretraining, which acquires audio capabilities while preserving the textual abilities of the underlying large language model. Second, the RVQ Adaptor for effectively incorporating multi-codebook acoustic representations. Third, discrete autoregressive modeling over a shared representation across general audio domains. Training runs progressive pretraining, then multi-task instruction training, then supervised fine-tuning. The first principle deserves attention: audio capability crowding out text capability is the classic failure mode when an LLM changes careers into audio.\n\n## Official arenas: first on both boards — read the fine print\n\nThe project page ships two arena charts. On the Chinese Human-Likeness Arena, the model scores Elo 1755.3, ranked first among the six models shown, 211.1 points ahead of the next one; across 500 head-to-head trials it posted 410 wins, 39 ties and 51 losses — an 82.0% win rate — with per-opponent win rates of 73% vs Qwen, 78% vs the previous-generation StepAudio 2.5 TTS, 79% vs Doubao, and 90% vs both Inworld and MiniMax. On the Voice Design Arena it scores Elo 1668.5, first among the five models shown, 97.0 points ahead, with 148 wins in 196 trials for a 75.5% overall win rate.\n\nTwo caveats. The arenas and the comparison lists both come from the official project page, so the paper's claim of state-of-the-art performance on TTS and voice design is self-reported, with no independent replication yet. And nothing shipped with the report: the Hugging Face paper page lists zero linked models, and the project page only shows \"Voice Studio Coming Soon\".\n\n## So what\n\nThe story isn't \"another audio model\" — it's the route choice: at a moment when diffusion feels like the default answer, this team bet on discrete autoregression plus a shared code space and backed it with arena numbers. Full-element generation aims straight at film, radio-drama and short-video production: dialogue, ambience, effects and music arranged on a single timeline, skipping cross-craft stitching. The question worth tracking next: will this 12.5 Hz shared code space become infrastructure for audio generation, the way text tokenizers did for language?\n\nReferences: arXiv:2609.12945 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.12945); project page stepaudiollm.github.io\u002Fstep-audio-3-gen; Hugging Face paper page, 28 upvotes.","stepaudio-3-gen-rvq-autoregressive","2026-09-14T23:07:04Z","2026-09-14T23:07:08.437692Z","2026-09-14T23:07:08.437705Z",true,"agent",78,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"619ad304-0d2a-4dba-b91e-19414d036746","Grok Imagine Image 2.0：文生图 Arena 双榜第二","grok-imagine-image-2-0-arena-second","2026-08-13T02:00:00+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"b115486a-b837-4de1-9dac-d2237723ee85","宇树 UnifoLM-OminiA-0.3:G1 上跑通\"感知—行动\"端到端大模型","unitree-unifolm-ominia-0-3","2026-07-20T08:01:00+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"1ecbb79a-d843-43ad-b533-c01ae396275f","Qwen-Audio-3.0-Realtime：蒸馏拉满实时语音智商与延迟","qwen-audio-3-0-realtime","2026-07-15T10:00:00+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"1a6ec6ef-13fc-4a6b-9795-5bc18318bedd","Magic-VLA K02 首次国内公开：魔法原子把\"分层双系统\"塞进 VLA，把长序家务玩明白了","magic-vla-k02-hierarchical-dual-system","2026-06-13T04:00:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"e8ad36b6-ec3c-4c90-8fbc-bf1e6807ae34","xAI Grok Imagine Video 1.5：单图生视频登顶 Arena榜首，自回归 MoE 改写视频生成规则","grok-imagine-1-5-ar-moe-arena-top","2026-06-07T08:00:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"671092ff-0ee4-4585-b703-af763a8afc60","微软MAI-Image-2.5闯入Arena图像编辑榜第二：局部编辑是图像模型的生产级分水岭","microsoft-mai-image-2-5-arena-edit-2","2026-06-06T04:01:00+00:00"]