[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-vibevoice-asr-streaming-speaker-attribution":3,"topics-all":35,"news-related-9bd3ef9b-d6b9-4793-9156-15ce7d91a66f":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"9bd3ef9b-d6b9-4793-9156-15ce7d91a66f","微软开源流式说话人 ASR:VibeVoice 7B 实时转录“谁在说什么”","微软研究院开源 VibeVoice-ASR-Streaming:基于 LLM 的端到端流式说话人转录模型,边听边输出“谁在说什么”,免独立说话人分离步骤。论文报告 7B 版五组评测平均 WER\u002FCER 最低,说话人归属 13 项设置中 12 项最佳,支持十语言与自定义热词,1.5B\u002F7B 权重加推理代码全部开源。","传统语音转写管线里,“听清说了什么”和“分清是谁在说”历来是两件事:ASR 模型负责前者,说话人分离(diarization)系统负责后者,两段管线拼起来,才得到一份带人名的会议纪要。这种拆分在离线场景没问题——反正音频录完再处理也来得及——但实时语音助手和 voice agent 等不起:等整段音频齐了再跑分离,延迟预算直接爆表。微软研究院 9 月 2 日提交到 arXiv 的技术报告给出了新解法:VibeVoice-ASR-Streaming,一个基于 LLM 的端到端流式说话人转录模型([arXiv:2609.02812](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02812))。\n\n## 边听边输出“谁在说什么”\n\n模型的核心机制,是把固定大小的音频块、少量前瞻(lookahead)音频与已生成的文本交错输入,让它在语音持续到达的同时输出“谁说了什么”,不再需要独立的 diarization 阶段。论文自称这是最早一批基于 LLM 的端到端流式说话人 ASR 方案——“自称”两个字值得保留,这类“first”式声明目前只有论文单一信源,按惯例等社区复现再完全采信。\n\n评测数据同样出自论文自报口径:7B 模型在五组评测集上取得最低的平均 WER\u002FCER;说话人归属在 13 项评测设置中拿下 12 项最佳或并列最佳。除 7B 外还有 1.5B 版本,两档权重连同推理代码全部开源,GitHub 仓库为 microsoft\u002FVibeVoice,Hugging Face 模型卡标注 MIT 许可证。\n\n工程侧还有两个实用设计:一是自定义热词,可以把人名、技术术语表注入模型,提升领域内容的识别准确率;二是十语言支持——中、英、法、德、意、日、韩、葡、俄、西,覆盖主流会议场景绰绰有余。\n\n## voice agent 为什么需要它\n\n这件事的真正分量在延迟预算里。实时 agent 的多轮对话中,如果说话人信息要等 diarization 后处理才能补上,上下文就出现断层:agent 分不清刚收到的这句话来自用户还是背景里的第三人。把归属判断做进流式解码本身,多人对话场景下的 agent 才具备“当场知道谁在说话”的能力——这正是语音交互从单人说、单人听走向真实多人环境的关键一步。1.5B 档的放出则给了低资源部署一个明确选项,毕竟不是所有边缘设备都塞得下 7B。\n\n从产品谱系看,VibeVoice 家族此前以长篇多说话人 TTS 闻名,这次是 ASR 侧的补位。微软把语音的“生成”与“理解”两端都押在开源上,姿态相当明确:语音 agent 基础件这块,想用生态打法吃下。\n\n## 所以呢\n\n对做语音产品的团队,这条新闻的价值不在跑分,而在“流式 + 说话人归属 + 热词”三件事被一个模型全包,还挂 MIT 许可。自托管实时会议转写、多人语音 agent 的对话记忆,现在都有可以直接下手的权重。留个心眼的只有一点:评测与“最早一批”均为官方自报,采购决策前先自己跑一遍评测集。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02812","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"279d1b31-7733-4740-945e-626131fcdb40","en","Microsoft Open-Sources VibeVoice-ASR-Streaming Speaker Transcription","Microsoft Research open-sources VibeVoice-ASR-Streaming, an LLM-based end-to-end model that streams who-said-what transcription without a separate diarization stage. The paper reports the 7B model achieves the lowest average WER\u002FCER across five evaluation sets and best or tied-best speaker attribution on 12 of 13 settings, with 10-language support, custom hotwords, and 1.5B\u002F7B weights plus inference code released under MIT.","In a traditional speech pipeline, transcribing what was said and figuring out who said it have always been two separate jobs: an ASR model handles the former, a speaker diarization system handles the latter, and only by stitching the two together do you get meeting minutes with names attached. Offline, that split is fine — the audio is finished anyway. Real-time voice assistants and agents cannot afford it: waiting for the full audio before running diarization blows the latency budget. A technical report submitted to arXiv on September 2 by Microsoft Research offers a new answer: VibeVoice-ASR-Streaming, an LLM-based end-to-end streaming speaker-attributed transcription model ([arXiv:2609.02812](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02812)).\n\n## Streaming who-said-what as speech arrives\n\nThe core mechanism interleaves fixed-size audio chunks, a small amount of lookahead audio, and previously generated text, so the model keeps producing who said what as speech arrives — with no separate diarization stage. The paper describes this as one of the first LLM-based end-to-end approaches to streaming speaker-attributed ASR. Keep the word “describes” in mind: this first-of-its-kind claim currently rests on a single source, and by usual practice deserves full credit only after community reproduction.\n\nThe evaluation numbers also come from the paper’s self-reported setup: the 7B model achieves the lowest average WER\u002FCER across five evaluation sets, and best or tied-best speaker attribution on 12 of 13 evaluation settings. Alongside the 7B model there is a 1.5B variant; both weight sets ship with inference code, the GitHub repository is microsoft\u002FVibeVoice, and the Hugging Face model card lists an MIT license.\n\nTwo engineering details stand out for practitioners. First, customized hotwords: names and technical term lists can be injected to improve recognition of domain-specific content. Second, ten-language support — Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish — enough coverage for mainstream meeting scenarios.\n\n## Why voice agents need this\n\nThe real weight of this work sits in the latency budget. In multi-turn real-time agent conversations, if speaker identity only arrives after a diarization post-process, the context breaks: the agent cannot tell whether the incoming utterance came from the user or a third person in the background. Baking attribution into streaming decoding itself is what lets agents in multi-speaker scenes know on the spot who is talking — a key step for voice interaction moving from one-speaker, one-listener setups into real multi-person environments. The 1.5B release also gives low-resource deployments an explicit option, since not every edge device can fit 7B.\n\nIn product-line terms, the VibeVoice family was previously known for long-form multi-speaker TTS; this release fills the ASR side. Microsoft is betting open weights on both the generation and understanding ends of voice — a clear ecosystem play for voice-agent infrastructure.\n\n## So what\n\nFor speech product teams, the value here is not the benchmark scores but the bundle: streaming, speaker attribution, and hotwords in one model, under an MIT license. Self-hosted real-time meeting transcription and multi-speaker voice-agent memory now have weights to grab directly. The one caveat: both the evaluations and the first-of-its-kind framing are vendor-reported — run your own eval set before making procurement decisions.","vibevoice-asr-streaming-speaker-attribution","2026-09-07T15:10:00Z","2026-09-07T15:11:22.133520Z","2026-09-07T15:11:22.133531Z",true,"agent",167,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"089f56f3-32ff-4036-89b5-728d5f5a9359","边聊边干活:腾讯混元开源全模态交互 Agent Gander,小脑管对话、大脑管执行","hunyuan-gander-omni-interaction-agent","2026-09-09T21:07:00+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"cd49f913-cde7-4cf3-8d93-24508653180e","腾讯混元开源AuK:1.5B语音模型统一生成与编辑,4步推理快4.5倍","tencent-hunyuan-auk-speech-editing","2026-09-09T09:12:00+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"ff0bc92a-295a-4707-be8d-76115fe9eeee","PerceptionBench 出炉:16 个前沿多模态模型,视觉感知无一及格","moonshot-perceptionbench-atomic-perception","2026-08-26T13:15:00+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"ce70384a-990b-4994-bfb6-27775be45661","TensorRT Edge-LLM 0.10.0：边端第一个统一的 C++ 多模态推理栈","tensorrt-edge-llm-0-10-multimodal-runtime","2026-08-23T00:00:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"f5c0b227-faf9-47e3-863f-3c102365cd41","LongCat-Next 开源：把文字、图像和声音统一成离散 Token","longcat-next-discrete-native-multimodal","2026-08-09T08:00:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"40095b51-97b0-4fd4-9b1d-f636c970572e","阿里 Qwen 团队发布 Qwen3.8-Max:2.4 万亿参数 MoE 模型首度开放权重","qwen3-8-max-2-4t-moe-open-weights","2026-08-07T02:00:00+00:00"]