[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-youtu-parsing-omni-osad-distillation":3,"topics-all":35,"news-related-4956a85c-3737-47be-b4a2-20d2b8e89fa0":64},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"4956a85c-3737-47be-b4a2-20d2b8e89fa0","优图解析论文:蒸馏自进化,音频项压过Gemini","腾讯优图为 5B 全模态解析模型 Youtu-Parsing-Omni 发布论文：单一编码器以共享权重处理文档、图像、音频、视频，OSAD 蒸馏让监督随模型逐轮进化。OmniDocBench v1.6 拿 96.96，OmniParsingBench 75.08 仅次于 Gemini-3-Pro。","腾讯优图的 5B 全模态解析模型 Youtu-Parsing-Omni，权重 10 月中旬就上了 Hugging Face 和 ModelScope，当时大家看到的是榜单数字。10 月 29 日，团队放出了完整技术论文（arXiv:2610.39301），把「怎么训出来的」讲透了——论文次日冲上 Hugging Face 每日论文榜第二，拿到 66 个赞。\n\n## 一个编码器：连坐标系都共用\n\n业界现成的全模态解析器大多用两座分别预训练的塔，像素流和音频流要等进了语言模型才相遇。优图的做法是把两座塔换成单个 Youtu-Omni-Encoder——从一个预训练文本语言模型初始化，先把「读文字」的底子打牢；像素和 log-mel 音频帧只经过薄薄一层模态专属 stem 就映射成 token，然后由共享的双向 Transformer 在统一的 (t, h, w) 位置编码下统一上下文化。视频帧的 t 来自帧序号，音频块的 t 来自时间偏移，两套模态共享同一根时间轴。\n\n关键的几层「融合层」会在每个音频-帧边界开新注意力窗口，让每一帧都能看见同一瞬间的声音——跨模态对齐在编码器内部就完成，不用等解码器事后重建。整套设计里只有 stem 和合并投影头依赖模态，感知参数几乎全部共享。\n\n## OSAD：老师就是上一轮的自己\n\n训练侧的招叫 OmniSchema-Aware On-Policy Distillation（OSAD）。前提是十类解析任务共享同一个 OmniSchema 输出协议，于是一套 token 角色目标能吃遍所有任务：学生模型采样自己的解析结果，通过准入门（JSON 合法、schema 一致、坐标语法、重复检查）的 rollout 由一个冻结的参考副本打分——这个副本额外能看到参考答案。每个 token 被区分为内容或结构，内容用 JS 散度、结构用高温前向 KL。每轮结束时学生的 LoRA 权重合并，学生变成下一轮的老师：解析器和它的监督共同进化，不需要手工设计奖励，也不需要单独训一个教师模型。\n\n## 榜单：96.96 居首，75.08 紧追 Gemini\n\n官方给出的结果表里，OmniDocBench v1.6 它以 96.96 Overall 居首，身后是 TeleOCR 1.2B 的 96.91、Gemini-3-Pro 的 92.91 和 GPT-5.2 的 86.59。更全面的多模态基准 OmniParsingBench 上，Gemini-3-Pro 以 77.44 居首，它拿 75.08 排第二、开源权重模型里最靠前；其中音频项 78.08 压过 Gemini-3-Pro 的 76.74，图表解析 95.02 也是表内最佳，几何、自然图像、视频等单项仍落后 Gemini。两个长尾项有亮点：化学结构 ChemOCR 的 Tani@1.0 一致率 54.2 居表首；乐谱转写 PDMX-Synth 的 CER 22.73，甚至略优于专用音乐模型 LEGATO 的 23.30，把 GPT-5.6（45.02）和 Gemini-3-Pro（54.26）甩开一大截。\n\n## 所以呢\n\n解析赛道正在从「每个任务堆一个专用小模型」转向「一个 omni 底座 + 统一输出协议」：坐标编码成专用 token、按 0-1000 网格相对取值，时间戳直接落到 HH:MM:SS，输出是一个带边界框和时间段的 JSON 信封——这正是 Agent 感知层要的「可引用证据」格式。工程侧也给得足，vLLM 0.19 插件、OpenAI 兼容接口和全部任务提示词都在仓库里。OSAD 这种「参考副本滚动刷新」的自蒸馏，对攒不起大规模标注的团队尤其有参考价值。短板同样清楚：视频理解单项还没过 Gemini，总榜仍是第二。论文见 [arXiv:2610.39301](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.39301)，代码在 github.com\u002FTencentCloudADP\u002Fyoutu-parsing。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.39301","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"dd4bcb44-cca9-423a-8c0e-cdaeb38cde68","en","Youtu paper: self-evolving distillation, top audio score","Youtu-Parsing-Omni paper: one encoder, 10 tasks; OSAD distillation co-evolves with the model; 96.96 OmniDocBench; 75.08 OmniParsingBench.","Tencent's Youtu Lab open-sourced the 5B omni-modal parsing model Youtu-Parsing-Omni on Hugging Face and ModelScope in mid-October, and at the time what everyone saw was leaderboard numbers. On October 29 the team released the full technical report (arXiv:2610.39301), explaining how the model is trained — and the paper climbed to #2 on Hugging Face's daily papers the next day, with 66 upvotes.\n\n## One encoder, one coordinate system\n\nMost existing omni-modal parsers use two separately pre-trained towers; the pixel stream and the audio stream only meet inside the language model. Youtu replaces both towers with a single Youtu-Omni-Encoder, initialized from a pre-trained text language model to lock in text reading first. Pixels and log-mel audio frames pass through thin modality-specific stems into tokens, then a shared bidirectional Transformer contextualizes them under one (t, h, w) positional encoding. A video frame's t comes from its frame index, an audio chunk's t from its time offset — both modalities share the same timeline.\n\nA few fusion layers open a new attention window at every audio-to-frame boundary, so each frame attends to the sound of the same instant. Cross-modal alignment is computed inside the encoder rather than reconstructed by the decoder. Only the stems and merger\u002Fprojector heads depend on the modality; nearly all perception parameters are shared.\n\n## OSAD: the teacher is your previous self\n\nThe training recipe is OmniSchema-Aware On-Policy Distillation (OSAD). Because all ten parsing tasks share the OmniSchema output protocol, one token-role-based objective covers every task. The student samples its own parses; rollouts that pass the admission gate (JSON validity, schema conformance, box syntax, repetition checks) are scored by a frozen reference copy that additionally sees the reference parse. Each token is classified as content or structure — content uses Jensen-Shannon divergence, structure uses forward KL at a higher temperature. At every round boundary the student's LoRA adapters are merged and the student becomes the next teacher: parser and supervision co-evolve, with no hand-designed reward and no separately trained teacher.\n\n## Scoreboards: 96.96 on top, 75.08 chasing Gemini\n\nIn the official results tables, Youtu-Parsing-Omni tops OmniDocBench v1.6 with 96.96 Overall, ahead of TeleOCR 1.2B at 96.91, Gemini-3-Pro at 92.91 and GPT-5.2 at 86.59. On the broader OmniParsingBench, Gemini-3-Pro leads at 77.44 while Youtu takes 75.08 — second overall and the highest-ranked open-weight model. Its audio score of 78.08 beats Gemini-3-Pro's 76.74, chart parsing at 95.02 is the table's best, and geometry, natural-image and video items still trail Gemini. Two long-tail results stand out: on ChemOCR chemical-structure recognition its Tani@1.0 agreement of 54.2 leads the table, and on PDMX-Synth sheet-music transcription its CER of 22.73 slightly beats the specialized music model LEGATO at 23.30, far ahead of GPT-5.6 (45.02) and Gemini-3-Pro (54.26).\n\n## So what\n\nDocument parsing is shifting from stacking specialized small models to one omni backbone with a unified output protocol: coordinates are encoded as dedicated tokens on a 0-1000 grid, timestamps land in HH:MM:SS, and the output is a single JSON envelope of bounding boxes and temporal segments — exactly the \"citable evidence\" format an agent perception layer needs. The engineering is generous too: a vLLM 0.19 plugin, an OpenAI-compatible server and all task prompts ship in the repo. OSAD-style self-distillation with a rolling reference copy is particularly instructive for teams that cannot accumulate large annotations. The shortboard is equally clear: video understanding still trails Gemini, and the overall rank is second. Paper: [arXiv:2610.39301](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.39301); code at github.com\u002FTencentCloudADP\u002Fyoutu-parsing.","youtu-parsing-omni-osad-distillation","2026-10-31T19:14:00Z","2026-10-31T19:14:31.638764Z","2026-10-31T19:14:31.638785Z",true,"agent",438,[36,46,55],{"slug":37,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":32,"created_at":44,"modified_at":45},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":32,"created_at":53,"modified_at":54},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":56,"tag_slug":56,"title_zh":57,"title_en":58,"intro_zh":59,"intro_en":60,"id":61,"is_active":32,"created_at":62,"modified_at":63},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":65},[66,71,76,81,86,91],{"id":67,"title":68,"news_slug":69,"published_at":70},"55663822-d49d-4c7e-a7c7-ba9770657c25","清华DBS动态分频:音源分离SI-SDR提升29%","dynamic-band-splitting-audio-separation","2026-10-21T15:02:40+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"a491d4db-2d5a-4723-b093-28d1512fd647","VLA-Δ:物理状态变化当监督信号,LIBERO 98.4","vla-delta-physical-state-transition","2026-10-20T19:07:27+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"d78ea8a8-bebb-4b94-b340-127eb2874a73","Ovis 全模态嵌入 3B:综合分领先 5.19","ovis-omni-embedding-3b","2026-09-23T21:08:45+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"12c67d52-17a2-4df5-8386-35d18ffd221a","JEPA-Anything:一套预测框架打通七个领域,湿实验也给了背书","jepa-anything-orthogonal-predictive-factorization","2026-09-19T23:10:37+00:00",{"id":87,"title":88,"news_slug":89,"published_at":90},"e840fad5-b3a1-47cd-ac68-679d5f635dc1","世界状态交给程序管:Programmable World Model 让视频模型只管渲染","programmable-world-model-persistent-state","2026-09-10T17:10:00+00:00",{"id":92,"title":93,"news_slug":94,"published_at":95},"dc2f4ead-963c-4a8e-bd41-400bebf83bb4","物理、几何、外观一个模型全包:Puffin-World 开源,相机 roll 误差低至 0.26°","puffin-world-native-3d-world-states","2026-09-06T19:09:41+00:00"]