[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-qwen-audio-3-1-asr-speaker-diarization":3,"topics-all":41,"news-related-d83c4910-6dce-4171-8a44-9f68c9f841af":60},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":27,"news_slug":34,"published_at":35,"created_at":36,"modified_at":37,"is_published":38,"publish_type":39,"image_url":14,"view_count":40},"d83c4910-6dce-4171-8a44-9f68c9f841af","Qwen-Audio-3.1-ASR开源:转写自带说话人","阿里千问在 arXiv 公开 Qwen-Audio-3.1-ASR 技术报告:音频语言模型把说话人分离做进转写,输出自带说话人标签、时间戳与重叠语音,覆盖 30 语言、16 中文方言。官方评测称 GigaSpeechBench 七语言错误率全最低,宏平均 15.09%,上代 22.50%。","转写稿里\"这句话是谁说的\",过去要靠级联系统拼:ASR 先出文字,再跑分离模型对齐说话人,会议一长标签错位、重叠语音丢失。阿里 Token Foundry 的千问团队 10 月 20 日把 Qwen-Audio-3.1-ASR 技术报告挂上 arXiv,把两步并成一步:模型直接输出带说话人标签和时间戳的结构化转写,重叠语音也保留在输出结构里。\n\n## 说话人分离长进了模型本身\n\n报告把这一代的能力扩展概括为两项:说话人分离转写与通用音频理解。前者输出\"谁、何时、说了什么\",跨对话轮保持标签一致;后者由 Next 变体承担,覆盖语音、环境声、音乐三类字幕生成,以及时间定位、声音事件检测和音频问答。家族仍支持 30 种语言和 16 种中文方言,保留历史上下文条件化、P0\u002FP1 两级热词定制、原生转写润色等指令控制能力。\n\n部署分五个变体:异步文件转写、低延迟流式、上下文语音输入、非流式识别与 Next。能力矩阵有明确取舍——流式与 Message ASR 不支持说话人分离和音频理解,这些能力只在非流式变体提供。\n\n## 两千万小时与六阶段训练栈\n\n训练分六个阶段:编码器预训练、音频 LLM 预训练、冷却适应、SFT、强化学习、多教师蒸馏。编码器用 BEST-RQ 在无标注语音上自监督预训练,部分层从预训练 Qwen3 LLM 初始化;音频 LLM 解码器骨干为 MoE,在约 2000 万小时多语言数据上预训练;冷却阶段用约 100 万小时高质量数据再平衡语言与方言。\n\n两个工程细节值得记:说话人 ID token 只占监督 token 的 2.90%、损失贡献仅 1.97%,团队因此做了损失重加权;时间戳相对 1 毫秒量化后,DER 与 cpWER 收敛显著加快。RL 用 FunVerl 框架,奖励含 1−WER、非对称热词奖励与时间一致性 tcpWER;蒸馏按任务路由到三个音频教师加 Qwen3.5 文本对话教师。\n\n## 跑分:方言是主场,MMSU 没赢\n\n先说归因:以下数字全部来自官方技术报告,Panel B 出自团队统一 API 评测管线。多语言侧,GigaSpeechBench 七种语言上该模型在对比系统里全部最低,宏平均 15.09%,上代为 22.50%;对比名单含 Doubao-ASR、腾讯云 ASR、Gemini 3.1 Pro、GPT-4o Transcribe 等。方言侧差距更大:公开基准宏平均 CER 4.55%,内部 16 方言测试集 10.38%,对比 Doubao 的 20.23% 与腾讯 Hy-ASR 的 17.13%。\n\n说话人分离转写上,两个变体在四个测试集 × DER\u002FWER\u002FcpWER 全部 12 个组合里集体最低,Next 在 AISHELL-4 的 DER 为 2.70,级联 Fun-ASR 基线全败。流式首字延迟 100 毫秒,对比 Doubao 的 570 毫秒——注意是自托管服务上测的。音频理解侧,MMAU 81.60% 超 Gemini-3.1-Pro 的 80.90%,五个专项全胜;但 MMSU 上 Gemini 的 85.30% 仍更高。\n\n## 单源报告,理性读\n\n按社区收录站 theopenweights 条目,权重已在 Hugging Face 放出,Apache 2.0 许可,参数规模未公布。全部跑分属官方自报,独立复现还没出现,引用具体数字时保留归因。\n\n所以呢:如果会议纪要、客服质检、访谈整理是你的业务,级联管道这次真的有了被单模型开源方案替代的迹象——但先确认你要的是非流式场景,流式 profile 目前拿不到说话人分离。\n\n参考:[arXiv:2610.28711](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28711)。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28711","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21,24],{"id":12,"name":13,"slug":13,"description":14,"color":14},"c4455801-0273-477d-9e92-00b4addfa439","asr",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":25,"name":26,"slug":26,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[28],{"id":29,"lang":30,"title":31,"summary":32,"content":33},"aa009800-3d93-4b50-aacc-4e19daab9468","en","Qwen-Audio-3.1-ASR: Speaker Diarization Built Into Transcription","Qwen-Audio-3.1-ASR: diarization built into transcription, 30 languages, 16 dialects. Lowest error on all 7 GigaSpeechBench languages; MMAU 81.6% tops Gemini.","\"Who said this line\" used to be bolted onto transcripts by a cascaded pipeline: an ASR model produces the words, then a separate diarization system aligns speakers. Long meetings mean misaligned labels and lost overlapping speech. Alibaba Token Foundry's Qwen team posted the Qwen-Audio-3.1-ASR technical report to arXiv on October 20, and the approach merges the two steps: the model directly emits structured transcripts with speaker labels and timestamps, and overlapping speech survives in the output structure.\n\n## Diarization moves into the model itself\n\nThe report frames this generation as two capability extensions: speaker-diarized transcription and general audio understanding. The former outputs who, when, and what, keeping speaker labels consistent across turns; the latter ships in the Next variant and covers captioning across speech, environmental sound and music, plus temporal grounding, sound event detection and audio question answering. The family still supports 30 languages and 16 Chinese dialects, and retains the instruction-controlled controls of the previous generation: historical-context conditioning, two-tier P0\u002FP1 hotword customization, and native transcription polishing.\n\nDeployment splits into five variants: asynchronous file transcription, low-latency streaming, context-aware voice input, non-streaming recognition, and Next. The capability matrix has explicit trade-offs — the streaming and Message ASR profiles do not support diarization or audio understanding; those live only in the non-streaming variants.\n\n## 20 million hours and a six-stage training stack\n\nTraining runs six stages: audio encoder pretraining, audio LLM pretraining, cooldown adaptation, SFT, reinforcement learning, and multi-teacher on-policy distillation. The encoder is first pretrained with BEST-RQ self-supervision on unlabeled speech spanning AI, biotech, e-commerce, education, entertainment, finance and mobility; some Transformer layers initialize from a pretrained Qwen3 LLM, and the non-streaming encoder swaps causal attention for bidirectional. The audio LLM decoder initializes from the language-model component of Qwen on a MoE backbone, pretrained on roughly 20 million hours of multilingual data with the encoder frozen; the cooldown stage then rebalances languages and dialects on about one million higher-quality hours, re-transcribing with VibeVoice and dropping samples with word error rate above 0.3.\n\nTwo engineering details are worth noting: speaker-ID tokens account for only 2.90% of supervised tokens and 1.97% of the unweighted loss, so the team applied loss reweighting; timestamp precision is quantized relative to 1 ms, which speeds up DER and cpWER convergence. The RL stage uses the FunVerl framework with rewards including 1−WER, asymmetric hotword rewards, cpWER, and time-consistent tcpWER; distillation routes examples to three audio teachers plus a Qwen3.5 text-and-dialogue teacher.\n\n## Benchmarks: dialects are the home turf, MMSU not won\n\nAttribution first: all numbers below come from the official technical report; Panel B results come from the team's unified API evaluation pipeline, with some baselines taken from public reports and some self-measured. On multilingual ASR, the model posts the lowest error rate among compared systems on all seven GigaSpeechBench East\u002FSoutheast Asian languages, macro-average 15.09% versus 22.50% for the previous generation; the comparison set includes Doubao-ASR, Tencent Cloud ASR, Gemini 3.1 Pro, GPT-4o Transcribe, Azure MAI-Transcribe-2, and Muse Voice Transcribe-1.0. Dialects show a wider gap: 4.55% macro-average CER on public benchmarks, and 10.38% on the internal 16-dialect test set, against 20.23% for Doubao and 17.13% for Tencent Hy-ASR.\n\nOn speaker-diarized transcription, the two variants collectively post the lowest rates across all 12 dataset-metric combinations (DER\u002FWER\u002FcpWER on four test sets); Next reaches DER 2.70 on AISHELL-4, while the cascaded Fun-ASR baseline loses on all four sets. Streaming first-character latency is 100 ms versus 570 ms for Doubao Streaming ASR 2.0 — measured, note, on a self-hosted deployment. On audio understanding, MMAU 81.60% edges past Gemini-3.1-Pro's 80.90%, it sweeps all five specialized tasks (acoustic scenes, non-speech vocal sounds, emotion, musical instruments, speaker tasks), and Clotho-Moment temporal grounding hits 82.20 mIoU, 10.41 points ahead of the runner-up; but Gemini-3.1-Pro keeps the lead on MMSU at 85.30%.\n\n## A single-source report; read it rationally\n\nPer the community tracker theopenweights, weights are on Hugging Face under Apache 2.0, with parameter counts undisclosed. All benchmark numbers are vendor-reported; independent replications haven't appeared yet, so keep the attribution when citing specifics.\n\nSo what: if meeting notes, call-center QA, or interview transcription is your business, this is the first credible sign that a single open-weights model can replace the cascaded pipeline — but check first whether you need streaming, because the streaming profile does not get diarization.\n\nReference: technical report [arXiv:2610.28711](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28711); community listing [theopenweights](https:\u002F\u002Ftheopenweights.com\u002Fnews\u002Fqwen-audio-3-1-asr-u6lx).","qwen-audio-3-1-asr-speaker-diarization","2026-10-22T17:13:29Z","2026-10-22T17:13:55.532562Z","2026-10-22T17:13:55.532569Z",true,"agent",14,[42,51],{"slug":43,"tag_slug":43,"title_zh":44,"title_en":45,"intro_zh":46,"intro_en":47,"id":48,"is_active":38,"created_at":49,"modified_at":50},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":52,"tag_slug":52,"title_zh":53,"title_en":54,"intro_zh":55,"intro_en":56,"id":57,"is_active":38,"created_at":58,"modified_at":59},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":61},[62,67,72,77,82,87],{"id":63,"title":64,"news_slug":65,"published_at":66},"35e9f682-7952-4f6a-a8e9-7f8bdd47e791","IBM 语音三件套更新:2B 旗舰 ASR 精度开源第二","granite-speech-5-collection-update","2026-10-22T21:08:48+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"405b8f47-8862-4684-ab0e-d552ce093072","MJev-Spatial:空间理解不生成文字,单次前向快5.6倍","mjev-spatial-classification-spatial-intelligence","2026-10-18T17:14:25+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"cb64371f-62b6-473d-8150-b576001d3f56","Qwen3.8-27B 开源权重上线:单卡跑得动的 Qwen3.8,还塞了个视觉编码器","qwen3-8-27b-open-weights-release","2026-08-14T19:30:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"40095b51-97b0-4fd4-9b1d-f636c970572e","阿里 Qwen 团队发布 Qwen3.8-Max:2.4 万亿参数 MoE 模型首度开放权重","qwen3-8-max-2-4t-moe-open-weights","2026-08-07T02:00:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"bbbda7ff-b1c7-4142-b37f-2b3fd0547e02","Fibo 2.0 开源:生成编辑合一,登顶开源编辑榜","bria-fibo-2-0-unified-editing-generation","2026-10-21T21:08:11+00:00",{"id":88,"title":89,"news_slug":90,"published_at":91},"bef7a773-36fb-406b-949d-8d728b97b242","Matrix-Game 4.0 开源:单卡 60FPS 的实时可玩游戏世界","matrix-game-4-0-open-source-60fps","2026-10-21T13:09:32+00:00"]