Tencent's Youtu Lab open-sourced the 5B omni-modal parsing model Youtu-Parsing-Omni on Hugging Face and ModelScope in mid-October, and at the time what everyone saw was leaderboard numbers. On October 29 the team released the full technical report (arXiv:2610.39301), explaining how the model is trained — and the paper climbed to #2 on Hugging Face's daily papers the next day, with 66 upvotes.

One encoder, one coordinate system

Most existing omni-modal parsers use two separately pre-trained towers; the pixel stream and the audio stream only meet inside the language model. Youtu replaces both towers with a single Youtu-Omni-Encoder, initialized from a pre-trained text language model to lock in text reading first. Pixels and log-mel audio frames pass through thin modality-specific stems into tokens, then a shared bidirectional Transformer contextualizes them under one (t, h, w) positional encoding. A video frame's t comes from its frame index, an audio chunk's t from its time offset — both modalities share the same timeline.

A few fusion layers open a new attention window at every audio-to-frame boundary, so each frame attends to the sound of the same instant. Cross-modal alignment is computed inside the encoder rather than reconstructed by the decoder. Only the stems and merger/projector heads depend on the modality; nearly all perception parameters are shared.

OSAD: the teacher is your previous self

The training recipe is OmniSchema-Aware On-Policy Distillation (OSAD). Because all ten parsing tasks share the OmniSchema output protocol, one token-role-based objective covers every task. The student samples its own parses; rollouts that pass the admission gate (JSON validity, schema conformance, box syntax, repetition checks) are scored by a frozen reference copy that additionally sees the reference parse. Each token is classified as content or structure — content uses Jensen-Shannon divergence, structure uses forward KL at a higher temperature. At every round boundary the student's LoRA adapters are merged and the student becomes the next teacher: parser and supervision co-evolve, with no hand-designed reward and no separately trained teacher.

Scoreboards: 96.96 on top, 75.08 chasing Gemini

In the official results tables, Youtu-Parsing-Omni tops OmniDocBench v1.6 with 96.96 Overall, ahead of TeleOCR 1.2B at 96.91, Gemini-3-Pro at 92.91 and GPT-5.2 at 86.59. On the broader OmniParsingBench, Gemini-3-Pro leads at 77.44 while Youtu takes 75.08 — second overall and the highest-ranked open-weight model. Its audio score of 78.08 beats Gemini-3-Pro's 76.74, chart parsing at 95.02 is the table's best, and geometry, natural-image and video items still trail Gemini. Two long-tail results stand out: on ChemOCR chemical-structure recognition its Tani@1.0 agreement of 54.2 leads the table, and on PDMX-Synth sheet-music transcription its CER of 22.73 slightly beats the specialized music model LEGATO at 23.30, far ahead of GPT-5.6 (45.02) and Gemini-3-Pro (54.26).

So what

Document parsing is shifting from stacking specialized small models to one omni backbone with a unified output protocol: coordinates are encoded as dedicated tokens on a 0-1000 grid, timestamps land in HH:MM:SS, and the output is a single JSON envelope of bounding boxes and temporal segments — exactly the "citable evidence" format an agent perception layer needs. The engineering is generous too: a vLLM 0.19 plugin, an OpenAI-compatible server and all task prompts ship in the repo. OSAD-style self-distillation with a rolling reference copy is particularly instructive for teams that cannot accumulate large annotations. The shortboard is equally clear: video understanding still trails Gemini, and the overall rank is second. Paper: arXiv:2610.39301; code at github.com/TencentCloudADP/youtu-parsing.