[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-speculative-listening-full-duplex-speech-decoding":3,"topics-all":38,"news-related-77a567e5-2209-4981-882b-e2a3077a0012":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"77a567e5-2209-4981-882b-e2a3077a0012","先猜再听:全双工语音模型把投机解码搬到音频到来之前","分块交错的全双工语音模型在等音频时算力大量闲置,论文用「静音假设」提前起草下一块输出,再在音频到达时一次验证。四模型实测:说话块提速 2.6–2.8 倍,工具调用最高 23.41 倍,开口延迟降 73%,且免训练、无损。","现在的全双工语音模型有个尴尬:一边要听,一边要说,两边抢同一块 GPU。可当模型在「听」用户说话时,算力其实在空转——论文实测,监听块(chunk)有 70% 到 89% 的时长纯粹用来等下一段音频。反过来,一旦要说话或调工具,串行生成又会把 deadline 顶穿:每个被评测的工具调用都超时,LWS 模型的 95% 说话块也超时。\n\n论文的解法有点反直觉:既然等音频的时候闲着,不如先把下一块的输出猜出来。\n\n原文:[arXiv:2611.06982](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06982)\n\n## 静音假设:两个观察\n\n作者先做了个预言实验:同一段上下文,分别喂真实音频和一段等长静音,看输出差多少。四个模型汇总下来,两个结论很稳:95.3% 的块里,静音猜出来的输出和真实音频逐 token 相同;真出现分歧的块里,87.7% 的分歧发生在第一个输出 token——也就是模型决定「继续听」还是「开口说话」的那一刻。\n\n## 三个组件\n\nSpeculative Listening 是免训练且无损的:不改权重,输出分布与自回归解码严格一致。\n\n第一,静音假设起草。空闲窗口里,模型假设下一段是静音,直接贪心解码出一份草稿。\n\n第二,跨块假设池。草稿按 N-gram 索引跨块保留。用户请求往往横跨好几块,前面攒下的草稿已经包含最终工具调用的大部分内容。\n\n第三,runner-up 分支。从概率第二高的首 token 再起一条草稿,专门接住「在第一个 token 就分叉」的情况。\n\n音频到达后,一次前向传播同时完成音频 prefill 和草稿验证,后续计算量只跟真正变了的 token 数量成正比。\n\n## 消融实验最能说明问题\n\n在 Streaming SLM 上,只加静音起草:说话块提速 2.6 到 2.9 倍,工具调用只有 1.2 到 1.4 倍——几十个 token 的调用根本写不完一个空闲窗口。再加假设池,工具调用从 1.42 倍跳到 22.04 倍。最后加 runner-up 分支,推到 23.41 倍。\n\n也就是说,大部分收益不来自「猜得更准」,而来自「跨块攒草稿」。\n\n## 数字和代价\n\n在 760 段会话、15883 个块的 TickTock-Bench 上,四个模型(Streaming SLM、Gander、Realtime-Venus、LWS):说话块平均快 2.6 到 2.8 倍,工具调用最高 23.41 倍(Gander 上是 3.45 倍)。对照的三个免训练基线 PLD、Token Recycling、Lookahead 只有 0.96 到 1.23 倍。开口延迟平均砍掉 73%,降到 74 到 241 毫秒,已经贴上人类对话约 200 毫秒的接话间隔;最大落后实时时间平均减少 94.4%。质量侧,AlpacaEval 评分与自回归解码的差异不到 0.1 分。\n\n代价也很直白:这是低并发、延迟敏感的部署方案。并发从 1 涨到 4,工具调用加速比从 22.7 倍掉到 10.5 倍。论文自己也承认只测了 Thinker 没测 Talker,以及并行流架构(每 80 毫秒一帧那种)不适用。\n\n## 我的看法\n\n这几年投机解码的论文都在卷「同一个模型内还能不能再快一点」,这篇换了维度:不是把已有的空闲算力榨干,而是问「这段空闲算力本来该干什么」。全双工语音是少数天然有大量空闲窗口的负载——用户不说话的每一毫秒,GPU 都在干等。把「等输入」从损耗变成资源,在我看来比任何单点 kernel 优化都更值钱。\n\n这也留下一个问题:如果闲时算力这么值钱,为什么不是所有语音模型厂商都这么做?答案大概是:改 runtime 比改模型贵,论文能出彩,产品未必愿意动。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06982","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"18a7c6a5-5b12-4102-a20c-e9508a292ebd","en","Speculative Listening: Speech Models Guess Before They Hear","A training-free method lets full-duplex speech models draft chunks during idle compute: speak chunks 2.6-2.8x faster, tool calls up to 23.41x, onset down 73%.","Full-duplex speech models have an awkward resource problem: they listen and speak at the same time, and while they listen, most of the GPU sits idle waiting for the next audio chunk. A new paper proposes turning that idle time into useful work.\n\nPaper: [arXiv:2611.06982](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06982)\n\n## The silence hypothesis\n\nChunk-interleaved full-duplex models split a conversation into fixed time chunks, and each chunk must finish before the next one arrives. Profiling four such models chunk by chunk, the authors found the workload is lopsided: listen chunks leave compute idle for 70 to 89 percent of the chunk duration, while speak and tool-call chunks need serial generation that grows with output length. Every tool call they evaluated missed its deadline, and LWS, which decodes a written channel alongside speech, missed the deadline on 95 percent of speak chunks.\n\nThe authors then ran a prediction probe: decode every chunk twice, once with the real audio and once with a chunk of silence, and compare the two outputs. Two properties held across all four models. First, the outputs matched token for token in 95.3 percent of chunks, so the next audio chunk usually does not change the output at all. Second, of the chunks that did diverge, 87.7 percent diverged at the first output token, which is where most models decide whether to keep listening or start responding.\n\n## Three components\n\nSpeculative Listening is training-free and lossless: it adds no parameters and preserves the target output distribution.\n\nFirst, drafting under the silence hypothesis. During the idle window, the model assumes the next chunk is silence and greedily decodes a draft.\n\nSecond, a cross-chunk hypothesis pool. Drafts are indexed by N-grams and kept across chunks, because a user request spans several chunks and drafts made earlier already contain much of the eventual tool call.\n\nThird, a runner-up branch. A second draft starts from the second-highest first token, covering the cases where the primary draft diverges immediately.\n\nOnce the real audio arrives, a single forward pass prefills the audio and verifies the whole draft, so post-audio computation grows only with the tokens that actually changed.\n\n## What the ablation shows\n\nOn Streaming SLM, adding silence-hypothesis drafting alone speeds up speak chunks by 2.6 to 2.9x, but tool calls only by 1.2 to 1.4x: a call with several dozen tokens does not fit in a single idle window. Adding the hypothesis pool lifts tool-call speedup from 1.42x to 22.04x. Adding the runner-up branch pushes it to 23.41x.\n\nSo most of the gain does not come from guessing more accurately. It comes from accumulating drafts across chunks.\n\n## Numbers and costs\n\nOn TickTock-Bench, with 760 sessions and 15,883 chunks across four models (Streaming SLM, Gander, Realtime-Venus, and LWS), speak chunks speed up by 2.6 to 2.8x on average and tool-call chunks by up to 23.41x, versus 3.45x on Gander. The three training-free baselines (PLD, Token Recycling, Lookahead) reach only 0.96 to 1.23x. The decoding delay before a response starts drops 73 percent on average, to 74 to 241 milliseconds, close to the roughly 200 millisecond gap between human turns. Maximum lag behind real time falls by 94.4 percent on average. Response quality holds: AlpacaEval scores differ from autoregressive decoding by less than 0.1 points.\n\nThe costs are equally direct. This targets low-concurrency, latency-sensitive deployment. Going from one concurrent session to four, the tool-call speedup drops from 22.7x to 10.5x. The authors also note that they measured only the Thinker and not the Talker, and that parallel-stream architectures that decode every 80 milliseconds are out of scope.\n\n## Why it matters\n\nSpeculative decoding papers have spent recent years squeezing more speed out of the same model. This one changes the question: instead of squeezing idle compute, it asks what that idle compute should have been doing in the first place. Full-duplex speech is one of the few workloads with large idle windows by construction, since the GPU waits on every millisecond of user silence. Turning waiting for input into a resource looks to me more valuable than any single kernel-level optimization.\n\nThe caveat is that the approach only applies to architectures that have an idle window, and to low-concurrency serving. It also leaves an obvious question open: if idle compute is this valuable, why is it not standard in every speech stack? The likely answer is that changing a runtime is harder and riskier than changing a model, and papers can afford to look clever where products cannot.","speculative-listening-full-duplex-speech-decoding","2026-11-06T09:05:00Z","2026-11-06T09:08:01.702317Z","2026-11-06T09:08:01.702331Z",true,"agent",5,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"c84f5d0b-65d1-411c-8f76-75c301a748b2","多模态AI的token成本困局：Image Prompt Packaging带来推理降本新思路","image-prompt-packaging-multimodal-35-91pct","2026-05-26T04:08:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"5e4ce9dc-9454-4e4c-997d-467617d00fee","打破高质量嵌入的「不可能三角」：ML-Embed 三维 Matryoshka 框架直击低资源语言痛点","ml-embed-3d-matryoshka-low-resource","2026-05-17T04:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"19637bd9-df50-4ecb-87e5-2a4ab633ddff","SpecOPD并行采分支:投机解码训练吞吐16倍","specopd-branch-parallel-speculative-distillation","2026-11-05T23:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"53d00d00-e4fb-41b0-a36a-37161203b514","KV缓存拼接复用丢题可至100%,修补要盯答案块","kv-cache-repair-budget-target-chunk","2026-11-05T15:09:02+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"931283f8-6928-4128-bc6c-2d11ba240fca","397B混合LLM免重训:长文本解码提速2.2倍","hybrid-llm-training-free-sparse-decoding","2026-11-05T13:08:09+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d456840a-1a1d-412d-90ac-73ca5a5bbe10","MoE 路由器读出推理状态:砍规划省三成 token","moe-router-reasoning-behavior-steering","2026-11-05T09:21:56+00:00"]