[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-costa-compressed-speech-token-adaptation":3,"topics-all":38,"news-related-2983ae2f-a24f-40ae-a348-6f2c74443b19":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"2983ae2f-a24f-40ae-a348-6f2c74443b19","CoSTA压缩语音token:口语对话生成提速1.7倍","中科院自动化所给出口语对话提速方案CoSTA,保留原tokenizer与解码器,只给生成侧扩压缩词表,词表扩展只增加约百分之六的参数,同预算压力测试能撑近一倍轮次,推理时确定性展开,合成接口不变。论文报告中文生成token压到66.81%,速度约1.7倍,延迟降三成,质量持平。","语音对话模型有一笔绕不开的账：模型每说一秒话，就要逐个自回归吐出一串语音 token。主流口语对话模型（SDM）的生成与合成共用同一套语音 token 表，而这套表通常按 25 Hz 起步——Step-Audio 2 mini 用的 S3Tokenizer V2 就是 25 Hz、6,656 个语音单元，一秒语音对应 25 步自回归生成。对话越长，推理越贵，延迟越难压。\n\n换低码率 tokenizer 是常见思路，但那是伤筋动骨：码率一降，压缩质量与声音细节的权衡就来了，而且原 tokenizer 与解码器整条管线都要替换，旧模型基本白训。中国科学院自动化所团队 10 月 15 日挂上 arXiv 的 CoSTA（[论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.21143)）走了第三条路：不动原 tokenizer 和解码器，只给生成侧扩一套压缩词表——模型以后「说短话」，解码器照旧「听长话」。\n\n## 三步后置改造，只动约 6% 参数\n\n做法分三步。第一步词表扩展：把语音 token 序列当文本做 BPE，从对话语音 token 里学出 80K 个压缩单元扩进模型词表；新单元的 embedding 与输出层权重用其成员 token 的均值初始化，原始词表参数一个不动，词表扩展只增加约 6% 的模型参数。第二步发音适配：用 Emilia 中英各 300 小时语音，经 Seed-VC 统一音色后，教模型在压缩 token 空间里做语音合成。第三步对话适配：按 OpenS2S 的构造方式每语言约 5 万条语音对话（助手侧由 CosyVoice 2 用一致音色合成），只更新 LLM 参数，语音编码器、适配器、语音解码器全部冻结。推理时，模型吐出的压缩 token 被确定性展开回原始语音 token 序列，再喂给原封不动的解码器合成波形——压缩无损，合成接口不变。\n\n## 效率账：生成提速 1.7 倍，延迟砍三成\n\n论文在 vLLM 流式推理设定下测：CoSTA-ZH 把生成 token 压到 66.81%，相对速度 1.736 倍；英文侧 70.17%、1.659 倍。对照组 Vanilla-FT 用同样的适配数据、原 token 微调，压缩率仍是 100%，速度几乎不变——增益确实来自压缩 token 本身，不是多训了一遍。系统层面，中文全响应延迟降 32.8–39.0%，吞吐升 51.6–59.2%；英文分别降 22.4–26.0%、升 25.8–38.6%；峰值显存最多只多 0.68 GiB。质量侧，在 URO-Bench 的中英文数据集上，中文 G-Eval 从底座 69.24、Vanilla-FT 69.88 升到 72.27，ASR 错误率从 7.49% \u002F 4.96% 降到 1.51%，UTMOS 音质评分持平；每语言每指标 180 组配对人评里，流畅度 win-or-tie 达 86.1–96.1%，语义质量 83.3–87.7%。\n\n## 消融里的真信号\n\n最有信息量的是组件消融：去掉对话适配，G-Eval 从 72.27 直接崩到 24.35——只学会「压缩地发音」并不等于学会「用压缩 token 对话」，两阶段缺一不可。词表规模的权衡同样实在：在词表实验里，从 80K 加到 160K，压缩率从 67.50% 降到 63.62%，速度只再涨 1.1%，G-Eval 反而从 81.36 掉到 78.87，所以 80K 是论文选定的平衡点。限定生成预算的压力测试更直观：同样预算下中文能撑 7.21 轮有效对话（Vanilla-FT 3.71 轮，底座 2.14 轮），英文 5.79 轮（1.84 轮）。换到 OpenS2S 这个不同架构（Think–Talk 结构、12.5 Hz 的 GLM-4-Voice tokenizer）上，语音 token 步数仍降 21.17% \u002F 18.64%，质量还略升——说明这套后置改造可以与低码率 tokenizer 叠加使用。团队还测出压缩率随词表规模的对数近似线性下降（10K–320K 新增单元、八种配置）。\n\n## 泼冷水的部分\n\n所有跑分均出自论文与官方 GitHub 仓库自报，目前没有第三方复现；压力测试那组轮次数字，作者自己写明衡量的是「生成格式的可持续性」而非对话连贯性；效率结论也绑定 vLLM 流式推理设定。另外这篇 10 月 15 日就挂上了 arXiv，直到 10 月 30 日被提交进 Hugging Face 论文榜才进入社区视野——好工作被发现，有时也得等一个窗口。\n\n语音模型这一轮在卷参数、卷全双工，CoSTA 提醒还有一个更朴素的方向：让模型每秒必须吐的 token 变少。对延迟来说，最贵的 token 是你根本不必生成的那个。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.21143","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":22,"name":23,"slug":23,"description":14,"color":14},"045c011e-e2bb-45ce-bdd6-0c927f8a3b87","token-efficiency",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"19cb2951-d4fa-4578-8de2-f96e25e35852","en","CoSTA: Compressed Speech Tokens Speed Spoken Dialogue 1.7x","CASIA's CoSTA adds compressed speech tokens to spoken dialogue models; the paper reports 1.7x generation speed and lower latency, quality intact.","Spoken dialogue models carry a stubborn cost: for every second of speech, the model autoregressively emits a stream of speech tokens, one step at a time. Mainstream SDMs share one speech-token vocabulary between generation and synthesis, and that vocabulary typically runs at 25 Hz — Step-Audio 2 mini's S3Tokenizer V2 runs at 25 Hz with 6,656 speech units, so one second of speech means 25 autoregressive steps. Longer conversations mean costlier inference and harder latency targets.\n\nSwapping in a lower-rate tokenizer is the obvious fix, but it is invasive: compression-quality trade-offs appear, and the original tokenizer–decoder pipeline has to be replaced, which strands the trained model. CoSTA ([paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.21143)), posted to arXiv on October 15 by a team from the Institute of Automation, Chinese Academy of Sciences, takes a third path: keep the original tokenizer and decoder untouched, and extend only the generation side with a compressed vocabulary — the model learns to \"speak short\" while the decoder keeps \"listening long.\"\n\n## Three post-hoc steps, about 6% more parameters\n\nThe method has three stages. First, vocabulary extension: speech-token sequences are treated like text and compressed with BPE; 80K compressed units learned from dialogue speech tokens are added to the model vocabulary. Each new unit's embedding and LM-head row is initialized with the mean of its component tokens' rows; original vocabulary parameters stay frozen, and the extension adds only about 6% of the original parameter count. Second, pronunciation adaptation: 300 hours each of Chinese and English speech from Emilia, timbre-unified with Seed-VC, teach the model to do TTS in the compressed token space. Third, dialogue adaptation: roughly 50K speech-dialogue examples per language, constructed following OpenS2S with assistant speech synthesized by CosyVoice 2 in a consistent voice, update only the LLM parameters while the speech encoder, adaptor, and speech decoder stay frozen. At inference, the compressed tokens the model emits are deterministically expanded back into the original speech-token sequence and fed to the untouched decoder for waveform synthesis — the compression is lossless and the synthesis interface is unchanged.\n\n## The efficiency ledger: 1.7x generation, latency down by a third\n\nUnder a vLLM streaming inference setup, the paper reports CoSTA-ZH compresses generated tokens to 66.81% with 1.736x relative speed; the English variant reaches 70.17% and 1.659x. The control, Vanilla-FT, fine-tunes on the same adaptation data with original speech tokens: compression stays at 100% and speed barely moves — the gains come from compressed tokens themselves, not from extra training. At the system level, full-response latency drops 32.8–39.0% in Chinese and 22.4–26.0% in English; throughput rises 51.6–59.2% and 25.8–38.6% respectively; peak memory grows by at most 0.68 GiB. On quality, across URO-Bench's Chinese and English datasets, Chinese G-Eval rises from 69.24 (base) and 69.88 (Vanilla-FT) to 72.27, ASR error falls from 7.49% \u002F 4.96% to 1.51%, and UTMOS speech quality holds steady; across 180 paired human ratings per language and metric, win-or-tie rates reach 86.1–96.1% for fluency and 83.3–87.7% for semantic quality.\n\n## The ablation that carries the paper\n\nThe most informative result is the component ablation: remove dialogue adaptation and G-Eval collapses from 72.27 to 24.35 — learning to \"pronounce in compressed form\" is not the same as learning to \"hold a dialogue in compressed tokens.\" The vocabulary-size trade-off is equally concrete: in the vocabulary experiments, growing from 80K to 160K added units pushes the compression rate from 67.50% down to 63.62% but buys only 1.1% more speed while G-Eval slips from 81.36 to 78.87, which is why 80K is the paper's chosen balance. The limited-budget stress test is vivid: under the same generation budget, Chinese sustains 7.21 effective turns (Vanilla-FT 3.71, base 2.14) and English 5.79 (1.84). Transferred to OpenS2S — a different architecture with a Think–Talk design and a 12.5-Hz GLM-4-Voice tokenizer — CoSTA still cuts speech-token steps by 21.17% \u002F 18.64% with slightly better quality, showing the post-hoc recipe can stack on top of low-rate tokenizers. The team also reports an approximately log-linear decline of compression rate with vocabulary size across 10K–320K added units and eight configurations.\n\n## Where to be skeptical\n\nEvery benchmark number is self-reported by the paper and its official GitHub repository; there is no third-party replication yet. The authors themselves note the stress-test turn counts measure \"generation-format sustainability rather than conversational coherence,\" and the efficiency results are tied to the vLLM streaming setting. Worth noting too: the paper sat on arXiv since October 15 and only entered community view when it was submitted to the Hugging Face papers board on October 30 — good work sometimes has to wait for its window.\n\nWhile the field races on parameters and full-duplex capability, CoSTA points at a plainer lever: make the model emit fewer tokens per second of speech. For latency, the most expensive token is the one you never have to generate.","costa-compressed-speech-token-adaptation","2026-10-31T15:13:09Z","2026-10-31T15:13:14.905669Z","2026-10-31T15:13:14.905681Z",true,"agent",244,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"0a4281e9-69b1-43f0-9e7b-78f0cd7ea4c2","医疗小模型回答砍72.8%,准确率反升3.2点","peca-conciseness-qlora-medical","2026-11-15T15:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"6186bff0-6bbb-4970-a0bc-b7bfa93a8e49","像素空间压缩:16个token装下RAG检索到的文档","opticalrag-pixel-space-rag-compression","2026-11-13T03:20:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9bdf46a1-30e1-47a8-bb3f-bc82bd0a25e1","4-bit多模态量化:视觉先崩,语言还在","proquant-mllm-w4a8-edge-quantization","2026-11-08T07:28:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"88a727c6-311a-4cf4-b5c3-98d4251d956a","语音蒸馏踩雷:分离任务从+6.6掉到-1.7","route-dmd-speech-distillation-separation-collapse","2026-11-08T05:05:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"376751a1-de8e-4e5b-afdc-4e26d7574146","NVIDIA Long-WAM：机器人记住 19 秒，预测未来再出手","nvidia-long-wam-robot-context","2026-10-08T19:11:55+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"90d79d15-21c4-4804-a668-7343be31c92e","百度搭子 DuMate 把 Agent 的 Token 砍掉 75%：Harness 工程如何成为智能体的成本胜负手","baidu-dumate-harness-75pct-token-cut","2026-06-15T08:00:00+00:00"]