[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-japanese-tts-polyphony-ruby-opd":3,"topics-all":35,"news-related-291c06d9-3435-4644-af2a-43bf91b54024":64},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"291c06d9-3435-4644-af2a-43bf91b54024","日语多音字97.13%读对:注音扰动加纯文本蒸馏","日语汉字一字多音是TTS的老大难,通用多语种模型在JKYB基准上读音准确率只有七成到九成。百度深圳团队的Ruby-OPD两步破局:Ruby注音表示加字形扰动让发音可控,再用纯文本在线蒸馏把注音知识搬进裸文本推理,准确率推到97.13%,目标假名错误率砍半至2.02%,核心流程不烧配对语音。","日语里一个「秋」字,单念是 aki,放进「晩秋」读 shū,在古文「危急存亡の秋」里又读 toki。更极端的例子:《某科学的超电磁炮》标题里的「超電磁砲」,标准读音本是 chōdenjihō,作品却要求它念成英文借词「レールガン」(Railgun)。让语音合成模型把这类多音字念对,是日语 TTS 最顽固的坑之一。\n\n端到端 TTS 普及之后,这个坑反而更深了。传统级联系统先用文本前端把文字转成音素,发音可控,但汉字字形里携带的语义和韵律线索被一并丢掉;新一代零样本模型直接吃原始文本,自然度上去了,多音字错误却压不住。百度深圳团队 11 月 4 日挂在 arXiv 的论文 Ruby-OPD 给出了一组下限数据:在常用汉字读音基准 JKYB 上,通用多语种模型 Qwen3-TTS 1.7B 的读音准确率只有 72.10%,FireRedTTS3 73.29%,FishAudio S2 Pro 89.90%,目标词假名错误率最高飙到 23.68%。\n\n## 把注音通道焊进训练\n\n方案的前半解决可控性。团队借鉴日本书报在汉字上方标注假名的 Ruby 排版,定义了隐式(括号附录音)与显式(带边界标记)两种注音表示。但直接拿注音数据微调,模型根本学不会看注音——基座在原始文本上预训练,本来就会「猜」大多数读音,没有动力理会注音,推理时甚至把字形和注音一起读出来。对策是字形扰动:训练时随机把汉字遮掉、替换成无关字符、或换成读音完全不同的近义汉字(比如把片假名ワールド换成念 sekai 的「世界」),逼模型只能依赖注音。消融实验里还有个反直觉结果:把整句全转成假名,效果反而比只转目标词更差——汉字本身就是分词与语义的锚,整句剥掉等于自废武功。\n\n## 不烧配对语音的在线蒸馏\n\n方案的后半解决裸文本准确率。冻结的「老师」看全注音文本,「学生」看裸文本;中间状态由学生从纯高斯噪声在线 rollout 得到,不含任何来自真实语音的信息,学生被训练在同一状态下逼近老师的速度场。数据侧从 UniDic 词典种子词出发,用 Gemini 3.1 Pro 合成出约 68 万个单音句和 10 万个多音句——蒸馏核心只需要(文本,注音)对,配对语音仅在训练中穿插两成用来稳住音质。效果很硬:只用注音数据做 SFT,裸文本准确率反而从基座的 94.11% 掉到 92.32%;加上在线蒸馏后拉到 97.13%,目标假名错误率从 4.68% 砍半到 2.02%,句子级错误率 0.70%,在论文表 2 的八个模型里排最前。测试基准 JKYB 覆盖 13,536 个句子、2,136 个常用汉字、4,512 种官方认可读音,专门针对这类读音歧义。\n\n## 代价与看点\n\n诚实的部分也要说:声音克隆相似度 CAM++ 为 0.703,略低于基座的 0.728;音质指标 UTMOSv2\u002FDNSMOS 与教师模型持平。可控性和准确率的收益,是拿一点克隆相似度换来的。基座是开源的 Irodori-TTS-v4.1-Small,微调用的 774 小时对齐语音来自开源数据集 Emilia 的日语子集,整套方法工程上可复现。\n\n对中文场景,这套思路几乎可以平移:中文多音字(「重庆」的「重」、「银行」的「行」)与日语汉字读音歧义同源,「保留汉字表层 + 注音通道 + 扰动训练」的组合拳可以直接抄作业。更值钱的是那个普适信号——发音知识可以从纯文本蒸馏进模型,不必为每个语种再烧一批录音棚数据。当语音合成转向长尾语种竞争,谁先把「文本即教材」用透,谁就先跨过数据瓶颈。论文地址:https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06403","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06403","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"c4455801-0273-477d-9e92-00b4addfa439","asr",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"e65a3af0-1db1-423d-836c-21f3d4e91f9e","en","On-policy distillation reads Japanese polyphony at 97.13%","Kanji polyphony trips TTS (72-90%). Ruby-OPD: ruby annotation plus text-only on-policy distillation hits 97.13%, target PER halved to 2.02%.","A single Japanese Kanji tells you how deep the polyphony rabbit hole goes: 「秋」 is aki standalone, shū inside 「晩秋」, and toki in the classical phrase 「危急存亡の秋」. The title of *A Certain Scientific Railgun* pushes it further — 「超電磁砲」 is canonically chōdenjihō, yet the work demands the English loanword rendering 「レールガン」 (Railgun). Getting text-to-speech to read these correctly is one of the most stubborn problems in Japanese voice synthesis.\n\nEnd-to-end TTS has made the hole deeper. Cascaded systems first convert text to phonemes through a front-end: pronunciation is controllable, but the semantic and prosodic cues carried by the Kanji glyphs are stripped away. Newer zero-shot models consume raw orthographic text directly — naturalness improves, yet polyphone errors persist. Ruby-OPD, a paper posted to arXiv on November 4 by a Baidu Shenzhen team, quantifies the floor: on the Joyo-Kanji-Yomi benchmark (JKYB), general multilingual models reach only 72.10% reading accuracy for Qwen3-TTS 1.7B, 73.29% for FireRedTTS3, and 89.90% for FishAudio S2 Pro, with target-word Kana error rates climbing to 23.68%.\n\n## Welding an annotation channel into training\n\nThe first stage tackles controllability. Borrowing the Ruby typography that annotates Kanji with small Kana in Japanese books, the team defines implicit (parenthesized pronunciation) and explicit (boundary-marked) annotation formats. Fine-tuning on annotation data alone fails: pretrained on raw text, the base model can already guess most readings, so it has little incentive to attend to annotations and even reads out both the glyphs and the annotations at inference. The fix is orthography perturbation — during training, randomly mask Kanji spans, substitute unrelated characters, or swap in semantically similar words with entirely different pronunciations (replacing the Kana word 「ワールド」, wārudo, with 「世界」, sekai), forcing the model to rely on the annotation. One counter-intuitive ablation: converting the whole sentence to Kana performs *worse* than converting only the target word — Kanji glyphs are the anchors for segmentation and meaning, and stripping them sentence-wide guts the model's context.\n\n## On-policy distillation without paired speech\n\nThe second stage tackles raw-text accuracy. A frozen teacher reads fully annotated text; the student reads raw text. Intermediate states are rolled out online by the student from pure Gaussian noise, carrying zero information leakage from real speech, and the student is trained to match the teacher's velocity field at those states. Seeded from UniDic dictionary entries, Gemini 3.1 Pro synthesizes roughly 680k monophonic and 100k polyphonic sentences — the distillation core needs only (text, annotation) pairs, with paired speech interleaved at just 20% of steps to anchor audio quality. The results are decisive: SFT on annotation data alone *drops* raw-text accuracy from the base's 94.11% to 92.32%; adding on-policy distillation lifts it to 97.13%, halves target Kana error from 4.68% to 2.02%, and reaches 0.70% sentence-level error — the top accuracy in the paper's eight-model Table 2. The JKYB benchmark itself spans 13,536 sentences, 2,136 regular-use Kanji, and 4,512 officially recognized readings.\n\n## Costs and takeaways\n\nThe honest part: voice-cloning similarity (CAM++) sits at 0.703, slightly below the base's 0.728, while speech-quality metrics UTMOSv2\u002FDNSMOS stay on par with the teacher. The gains in controllability and accuracy cost a sliver of cloning similarity. The base model, Irodori-TTS-v4.1-Small, is open source, and the 774 hours of aligned SFT audio come from the Japanese subset of the open Emilia dataset — the whole recipe is reproducible in engineering terms.\n\nFor Chinese, the recipe is nearly transplantable: Chinese polyphones (the 「重」 in 「重庆」, the 「行」 in 「银行」) share the same root ambiguity as Japanese Kanji readings, and the combination of surface-form preservation, an annotation channel, and perturbation training carries over directly. The broader signal is worth more: pronunciation knowledge can be distilled into a model from plain text alone, without burning another round of studio recordings per language. As speech synthesis competition shifts to long-tail languages, whoever squeezes \"text as the textbook\" hardest crosses the data bottleneck first. Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06403","japanese-tts-polyphony-ruby-opd","2026-11-05T21:07:29Z","2026-11-05T21:07:33.855840Z","2026-11-05T21:07:33.855848Z",true,"agent",173,[36,46,55],{"slug":37,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":32,"created_at":44,"modified_at":45},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":32,"created_at":53,"modified_at":54},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":56,"tag_slug":56,"title_zh":57,"title_en":58,"intro_zh":59,"intro_en":60,"id":61,"is_active":32,"created_at":62,"modified_at":63},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":65},[66,71,76,81,86,91],{"id":67,"title":68,"news_slug":69,"published_at":70},"1208ec8f-77b0-476f-8309-720517d22d32","8B语音基座追平4倍大Omni:索尼把力气全花在数据上","sony-walrus-8b-speech-audio-foundation","2026-11-11T21:06:18+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"9ee90f89-5d06-40dc-aeed-e06ebf292b0a","DEFINE 给 TTS 加口音旋钮:换腔调不换嗓","define-accent-control-zero-shot-tts","2026-10-05T15:06:46+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"9da6602d-0877-42d6-858b-667138639063","合成先行、真声收尾:低资源TTS的信任加权新解法","trust-aware-low-resource-tts","2026-09-24T15:20:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"7726a377-f3e3-4fd1-93eb-6c4b075b685f","论文实测:统一模型自画隐藏状态,代词可视化最高94%","v-lens-unified-model-hidden-state-visualization","2026-11-20T05:30:00+00:00",{"id":87,"title":88,"news_slug":89,"published_at":90},"72e668b2-3720-480b-af8d-9d14280bdf64","3%的注意力头,撑起冻结LLM的语音识别","slam-asr-induction-heads-noise-scaling","2026-11-19T21:15:00+00:00",{"id":92,"title":93,"news_slug":94,"published_at":95},"0b895f86-1c24-4c60-9f1f-c2fd0da42b2c","PO降幻觉:提前闭嘴能复现58%到86%的收益","po-hallucination-gain-reproduced-by-early-stopping","2026-11-19T01:30:00+00:00"]