Deep research agents are no longer novel: give one a question and it searches the web, writes a report, and adds citations. Swap the subject for a single video, though, and the task gets much harder — half the evidence sits inside the footage and its audio, half lies outside it, and the two halves must be reconciled. On October 22, a team from Peking University and Alibaba's Token Hub (first author an intern with the Qwen team) released OmniDR-Bench on arXiv, built to test exactly this: video-centered deep research reports. The outcome is not flattering. Across 18 model configurations, the highest point estimate — Claude Opus 5 — reaches only 63.3.

The task and the rubric: 6,414 atomic criteria

The task is direct: an agent receives a source video and a user request, must derive its own research questions, retrieve external evidence from webpages, images, videos and documents, and produce an interactive, cited HTML report anchored to the video timeline. In the authors' framing, existing benchmarks cover either text-only deep research or video question answering with short answers; none evaluated producing a full research report organized around one given video.

Scoring uses their Omni-ART atomic rubric trees: three dimensions, 23 shared directions, and task-specific binary criteria — 6,414 criteria in total — covering audiovisual grounding, multimodal evidence use, and interactive presentation. Construction ran through independent evidence collection, automated checks, and two rounds of human review; the candidate pool shrank from 82 tasks to 60 and then to the final 50 (bilingual Chinese-English, 19 real-world subcategories). A final audit sampled 1,000 criteria, with more than 94% judged correct. GPT-5.6 Sol serves as judge, paired with deterministic browser probes; 900 runs yielded 835 evaluable reports.

Presentation 81.8, content 45.4

Twelve models run on the Claude Code harness, with Codex and OpenClaw each covering three configurations. Three rows stand out. Claude Opus 5 tops the board at 63.3 but misses 4% of tasks. GLM-5.3 scores 62.0 while delivering all 50 tasks; the 1.3-point gap falls inside a confidence interval of [-4.0, 5.6], so the ranking is uncertain. Qwen3.8-Max posts the highest single report at 91.5, yet an 88% completion rate drags its overall score to 52.9.

The dimension breakdown stings more: under a shared report template, presentation averages 81.8, content 45.4, citation 47.8. Agents retell the video itself faithfully (direct-video-fact pass rate 79.9%, video faithfulness 92.4%), but step outside the video and they fall apart: external evidence use 8.7%, cross-source verification 8.8%, multi-fact composition 5.0%, claim support just 15.9%. The citation pattern is telling — GPT-5.6 Sol passes 98.0% of source-count criteria but only 23.9% of claim-support criteria; for Gemini-3.8-Flash the pair is 93.0% versus 13.9%. Links pile up in a final Sources section while the claims themselves sit unsupported.

Orchestration and harnesses: quality bought with money

The skill ablation prices every layer: interface-only baseline 28.0; adding video understanding 31.8; adding web research jumps to 45.5 (task success climbs from 73.3% to 100%); multimodal retrieval 46.6; full research orchestration 53.9. Full orchestration adds 7.3 points over all atomic skills (12 of 15 matched tasks improve, citation up 20.7), at the cost of median steps rising from 103 to 167, time from 0.7 to 1.6 hours, and tokens from 2.3M to 4.7M — a quality-cost trade, explicitly priced.

Harness choice matters too. The same GPT-5.6 Sol scores 54.9 on Claude Code, 53.9 on Codex, and only 41.7 on OpenClaw; swap in Qwen3.7-Max and Codex is suddenly best (45.3 versus 39.9). The two harnesses average an identical 48.4 across three matched models, but Codex completes 99.3% of runs against Claude Code's 96.0%. There is no universal winner — model and harness must be chosen together.

So what

The cold water this benchmark pours on deep-research hype is specific: hand agents a template and presentation scores soar; the actual research behaviors — finding external evidence, cross-checking sources, composing multiple facts — barely exist in any configuration. For buyers: read the content and citation scores, not the rendered demo. For builders: GLM-5.3's 100% delivery at 62.0 may be worth more in production than 63.3. The benchmark, code and trajectories are open — see arXiv:2610.31158 and the GitHub repo.