[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-marimba-typed-speech-decision-frozen-asr":3,"topics-all":38,"news-related-09df4ac8-eede-4a9c-b488-0a577b589417":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"09df4ac8-eede-4a9c-b488-0a577b589417","0.6B语音模型不生成答案,只回答是非题","论文 Marimba 冻结 Qwen3-ASR 0.6B 编码器,只挂几个小读出层,把音频窗口变成带概率的判断题。轮次结束 AUROC 0.935,高于 Qwen3-Omni 的 0.786;但在 Smart Turn v3.2 自己的测试集上是 0.930 对 0.982,输给了专用模型。","一个 0.6B 的语音识别模型，底座冻结不动、只在上面挂几个小读出层，在「这句话是不是该我接」这类问题上给出了比 30B 音频大模型更高的分数——这是论文自己那份对照表里的数字（https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18330）。有意思的地方不在这个分数，而在它背后那套完全不同的问法。\n\n## 决策，不是生成\n\n论文叫 Marimba，作者 Solano Amadori，署名 charcot.ai，2026 年 11 月 13 日提交 arXiv。核心机制只有一句：一个 8 秒音频窗口，冻结的 Qwen3-ASR 0.6B 编码器只前向一次，同一批特征复用来回答这个窗口上的多个问题。\n\n输出被限定成三种类型：choice、yes\u002Fno、score，都带概率，不带生成的解释文本。\n\n分工也很清楚。词面问题（有没有说出某个词）根本不做任务训练，直接看冻结解码器的输出：贪心解码加四条 beam 搜索共 5 条假设，归一化后按精确 token 匹配算比例，再用 Platt 缩放校准成概率。非词面问题走一个 question-conditioned head，用交叉注意力给每个候选项打分。轮次结束另开一路，在第 18 层特征上池化最后 2 秒。语言识别更省事，直接用 ASR 自带的语言 token 概率，覆盖 30 语种。说话人相关的三个判断挂在第 12 层。\n\n## 赢在哪，输在哪\n\n先说清楚一件事：论文那 1000 道固定公共问题的对照里，轮次结束这一行对 Marimba 是分布内的，因为它的读出层就是拿 Smart Turn 的训练数据训的；其他所有行对所有系统都是零样本。所以下面那个「赢了」要打折看。\n\n对照表里有两行值得单独拎出来。FLEURS 二元词检测，它的 AUROC 是 0.901，Qwen3-Omni 是 0.791，Qwen2-Audio 是 0.682。轮次结束那一行是 0.935 对 0.786 对 0.476——Qwen2-Audio 在这项上按随机水平走，95% 置信区间是 0.40 到 0.55，200 条样本里它全选了肯定项。\n\n但反过来也有两行是 Qwen3-Omni 领先：Speech Commands 二元 AUROC 0.991 对 0.926，Common Voice 语言识别准确率 1.000 对 0.904。官方仓库后来把对照表补到四个系统，加进来的 FireRedAudio 在轮次结束上只有 0.542。\n\n跟专用模型比才是更硬的考验。在 Smart Turn v3.2 自己的 500 条测试集上，Marimba 的读出层 AUROC 是 0.930，Smart Turn 是 0.982，配对差 -0.052，95% 区间从 -0.075 到 -0.032。换成域外的 ICSI 数据，双方是 0.716 对 0.712，差值 +0.004，区间 -0.046 到 +0.054——论文明确写了这是一次统计打平，既不算优势也不算等价。\n\n说话人那几项成绩不错。换人检测 0.966 和 0.980，ECAPA 基线是 0.924 和 0.962，pyannote 3.1 是 0.865 和 0.812；同一人 0.998 和 0.996，对 ECAPA 的 0.985 和 0.995；人数准确率 0.778 和 0.859，对 pyannote 的 0.738 和 0.836。但三人以上会少数——VoxConverse 上 667 个这类窗口里有 395 个被报成两人。语音活动在 AMI 上 0.986 到 0.995，VoxConverse 至少 0.999。\n\n词面和隐私这两块要一起看。语言识别在 FLEURS 上 300 条开集闭集都是 100%，Common Voice 闭集 93.33%、开集 84.33%。个人信息检测用了 200 条合成巴西葡萄牙语语音，词级 F1 0.904，100 条负样本里误报 3 条；换成参考转写则 F1 是 1.000，说明漏检来自转写本身——电话号码召回 0.70，CPF 号只有 0.60。\n\n## 论文自己把输的地方写清楚了\n\n域外场景分类这一项全在随机水平。四选一准确率 0.24 到 0.29，随机是 0.25；二元 AUROC 0.48 到 0.54。这说明这套读出层不是万能的通用语义头。\n\n延迟也不平均。轮次结束 13 毫秒，语音活动 16 毫秒，语言识别 161 毫秒，词检测要 2.35 秒——词面问题必须跑解码，编码器复用不等于流式就绪。跨域校准也别想捡便宜：在 ICSI 上做五折跨会议重校准能把 ECE 从 0.200 压到 0.011，但那是每折 200 条标注窗口换来的，论文自己说明这只说明域内，不等于跨域校准。\n\n工程细节上，官方仓库（https:\u002F\u002Fgithub.com\u002FCharcot-AI\u002Fmarimba-api）写明个人信息检测需要调用方自带文本检测器、仓库不提供，音频场景分类那栏直接标 unavailable，返回 None 而不是猜一个。\n\n伦理那段值得单独看一眼。私有数据来自一家商业临床记录平台，按该平台条款由临床医生接受并授权去标识化后用于训练，适用巴西 LGPD；但这项研究没有经过伦理委员会审查，患者也没有给研究专门的研究知情同意。私有数据不公开，只报聚合结果。\n\n## 这条路线值不值得跟\n\n机制层不是一篇论文的孤例。另一个团队的 ASOM 论文（https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.22401）走的是同一条路：冻结的语音与文本编码器接到一个 620 万参数的读出层，一次前向答完 20 个问题要 13.1 毫秒，比音频大模型快 17 到 41 倍。而轮次检测赛道的老对手 Smart Turn v3.2 本身也就约 800 万参数，基于 Whisper Tiny，某些 CPU 上 10 毫秒就能跑完（https:\u002F\u002Fgithub.com\u002Fpipecat-ai\u002Fsmart-turn）。\n\n底座也不是瓶颈。Qwen3-ASR-0.6B 以 Apache-2.0 开源，官方仓库写明覆盖 52 种语言和方言、并发 128 时达到 2000 倍吞吐（https:\u002F\u002Fgithub.com\u002FQwenLM\u002FQwen3-ASR）。冻结一个够用的底座再挂各种读出层，这事的成本已经低到不像个研究题目。\n\n我的判断是，语音 Agent 真正难的不是听懂，而是决定「现在该不该说话」以及「这个决定有多确信」。把问题写成是非题并强制输出概率，至少让上层能设阈值、能拒答——在实时语音栈里，这比多几个点的平均准确率更值钱。代价同样实在：词面问题因为要解码，延迟从 13 毫秒跳到 2.35 秒。\n\n所以问题留给你：当一个系统把「我不确定」变成一个可以编程的返回值时，你会更愿意把决策权交给它，还是更担心它用一个漂亮的概率把不确定性藏起来？","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18330","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"c4455801-0273-477d-9e92-00b4addfa439","asr",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"15a01533-47c7-478b-bc48-067a48858c05","en","Marimba Answers Voice With Probabilities, Not Text","A frozen Qwen3-ASR 0.6B encoder turns audio windows into typed probability questions: turn-end AUROC 0.935 beats Qwen3-Omni, loses to Smart Turn v3.2.","A 0.6B speech recognition model, with its backbone frozen and a few small readouts layered on top, scored higher than a 30B audio language model on questions like 'is this my turn to speak'. That is a number from the paper's own comparison table (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18330). What makes it interesting is not the score but the entirely different way of asking underneath it.\n\n## Decisions, not generation\n\nThe paper is called Marimba, authored by Solano Amadori, affiliated with charcot.ai, submitted to arXiv on 2026-11-13. The core mechanism is one sentence: for an 8-second audio window, a frozen Qwen3-ASR 0.6B encoder runs forward once, and the same features answer every question asked about that window.\n\nThe output is constrained to three types: choice, yes\u002Fno, and score, each carrying a probability, with no generated rationale attached.\n\nThe division of labour is explicit. Lexical questions, meaning 'was this particular word spoken', get no task-specific training at all: they read the frozen decoder's output directly, greedy decoding plus four beam searches for five hypotheses in total, normalized, matched as exact token sequences, then mapped to a probability through Platt scaling. Non-lexical questions go through a question-conditioned head that cross-attends to each candidate option. Turn-end detection takes a separate path, pooling the final two seconds of layer-18 features. Language identification is cheaper still, using the ASR model's own language token probabilities across 30 languages. The three speaker decisions sit on layer-12 features.\n\n## Where it wins, where it loses\n\nOne caveat first: in the paper's comparison over 1,000 fixed public questions, the turn-end row is in-distribution for Marimba, because its readout was trained on Smart Turn training data. Every other row is zero-shot for every system. So the wins below come with that discount.\n\nTwo rows of the comparison table deserve pulling out. On FLEURS binary term detection, its AUROC is 0.901 against 0.791 for Qwen3-Omni and 0.682 for Qwen2-Audio. On turn end the figures are 0.935, 0.786, and 0.476, with Qwen2-Audio sitting at chance: its 95% confidence interval runs from 0.40 to 0.55, and across all 200 clips it picked the affirmative option every time.\n\nTwo other rows go the other way, to Qwen3-Omni: Speech Commands binary AUROC of 0.991 against 0.926, and Common Voice language identification accuracy of 1.000 against 0.904. The official repository later extended the comparison table to four systems, adding FireRedAudio, which reached only 0.542 on turn end.\n\nA tougher test is a purpose-built model. On Smart Turn v3.2's own 500-clip test set, the Marimba readout reaches AUROC 0.930 while Smart Turn reaches 0.982, a paired difference of -0.052 with a 95% interval from -0.075 to -0.032. Out of domain on ICSI, the two are 0.716 and 0.712, a difference of +0.004 with an interval from -0.046 to +0.054, and the paper states plainly that this is a statistical tie, evidence for neither advantage nor equivalence.\n\nThe speaker results are good. Speaker change reaches 0.966 and 0.980, against an ECAPA baseline of 0.924 and 0.962 and pyannote 3.1 at 0.865 and 0.812. Same speaker reaches 0.998 and 0.996 against ECAPA at 0.985 and 0.995. Speaker count accuracy is 0.778 and 0.859 against pyannote at 0.738 and 0.836. But three or more speakers get undercounted: on VoxConverse, 395 of 667 such windows were predicted as two people. Voice activity runs 0.986 to 0.995 on AMI and at least 0.999 on VoxConverse.\n\nLexical and privacy results need reading together. Language identification hits 100% on both open and closed set over 300 FLEURS clips, and 93.33% closed set and 84.33% open set on Common Voice. Personal-information detection used 200 synthetic Brazilian Portuguese clips and reached item-level F1 of 0.904, flagging 3 of 100 negatives; on reference transcripts the F1 is 1.000, which shows the misses come from transcription itself, with recall at 0.70 for phone numbers and 0.60 for CPF numbers.\n\n## The paper writes down where it loses\n\nUnseen-task scene classification sits entirely at chance. Four-way choice accuracy is 0.24 to 0.29 where chance is 0.25, and binary AUROC is 0.48 to 0.54. These readouts are not a general-purpose semantic head.\n\nLatency is uneven too. Turn end is 13 ms, voice activity 16 ms, language identification 161 ms, and term detection 2.35 s, because lexical questions must decode, and encoder reuse does not establish streaming readiness. Cross-domain calibration is no bargain either: five-fold cross-meeting recalibration on ICSI brings ECE down from 0.200 to 0.011, but that costs 200 labelled windows per fold, and the paper itself notes this shows in-domain calibration only, not cross-domain calibration.\n\nOn engineering details, the official repository (https:\u002F\u002Fgithub.com\u002FCharcot-AI\u002Fmarimba-api) states that personal-information detection requires a caller-supplied text detector which the repository does not include, and that the audio scene classification lane is marked unavailable, returning None rather than guessing.\n\nThe ethics section is worth a separate look. The private data came from a commercial clinical documentation platform whose terms of use, accepted by the clinicians acting as data controllers under Brazil's LGPD, authorize the provider to de-identify the data and use it to train AI models. But the study was not reviewed by a research ethics committee, and patients did not give study-specific research consent. The private recordings are not released, and only aggregate results are reported.\n\n## Is this direction worth following\n\nThe mechanism is not one paper's lonely idea. A separate team's ASOM paper (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.22401) takes the same route: frozen speech and text encoders feeding a 6.2M-parameter reader, answering 20 questions in 13.1 ms, 17 to 41 times faster than the audio language models. And the incumbent in turn detection, Smart Turn v3.2, is itself roughly 8M parameters on a Whisper Tiny backbone, running in as little as 10 ms on some CPUs (https:\u002F\u002Fgithub.com\u002Fpipecat-ai\u002Fsmart-turn).\n\nThe backbone is not the bottleneck either. Qwen3-ASR-0.6B is released under Apache-2.0, and the official repository documents language identification and ASR for 52 languages and dialects, reaching 2000 times throughput at a concurrency of 128 (https:\u002F\u002Fgithub.com\u002FQwenLM\u002FQwen3-ASR). Freezing a good enough backbone and hanging readouts off it has become cheap enough to stop looking like a research problem.\n\nMy read is that the hard part of a voice agent is not understanding speech but deciding whether to speak now and how confident that decision is. Writing the question as a typed yes\u002Fno and forcing a probability out gives the layer above something it can threshold and refuse on, and in a real-time voice stack that is worth more than a few points of average accuracy. The cost is equally concrete: because term questions have to decode, latency jumps from 13 ms to 2.35 s.\n\nSo the question I leave with you: when a system turns 'I am not sure' into a programmable return value, are you more willing to hand it the decision, or more worried that it is using a tidy probability to hide the uncertainty?","marimba-typed-speech-decision-frozen-asr","2026-11-17T01:00:00Z","2026-11-17T01:20:06.357642Z","2026-11-17T01:20:06.357656Z",true,"agent",129,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"abd8eb9f-c106-4899-bf24-b6bbed3da5e3","EPFL论文实测:注意力砍维度而非砍token,只算三成点积","hisa-hidden-dimension-sparse-attention-rope","2026-11-20T09:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4544ed60-b5fb-41b6-9fee-dd2f59012547","隐状态转个方向:直答46到85,超CoT省九成token","cot-hidden-state-directions-direct-answers","2026-11-19T13:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9af27b13-b1a9-47e3-a143-fd8a6c1ee5a9","Gemma 4省下的37.5%,推理引擎还没兑现","gemma-4-kv-cache-k-equals-v-compact-row","2026-11-18T11:40:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"d9243227-680a-4fde-908d-8990969a54e0","LuthorSQL：7B 模型替 Agent 省三成账单","luthorsql-text-to-sql-agent-cascade","2026-11-16T05:00:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"f08a17d3-2856-46ca-a104-27174e709b8a","TTS模型自己当编码器:参数省48%,三基准双胜","rse-tts-shared-reference-encoding","2026-11-15T19:40:00+00:00"]