MiniMax-H3 is one of those "generates everything" models: text, images, video and audio jointly modeled in a shared latent framework. Since its open-weight release in late July it has been distilled (FastH3), post-trained (fal's H3 Max) and torn down by third parties. But a more fundamental question had no systematic answer: can it actually reason about the physical world? A 13-author team from the National University of Singapore, Fudan University and Tencent just published a 517-instance "physical exam" (arXiv:2609.18323), and the verdict is blunt — an overall success rate of 41.97%.

What makes this eval nasty: every single modality gives only half the evidence

By the paper's own account, most existing video-generation benchmarks share a flaw: prompts closely match the target content, so the model is mostly doing caption-to-video. This eval inverts that. The text prompt only specifies the operation to perform; task-relevant information is deliberately left unstated and must be recovered from complementary image, audio and video evidence. In the audio scenario, an image may contain several candidate sound sources — the model must first bind the supplied sound to the correct object, then express that interpretation in the generated output.

Four scenarios, 29 subcategories:

  • Multi-view Spatial Reasoning (MSR) — single image, 200 instances, spatial relationships across complementary views: 43.50%
  • Audio-based Disambiguation Reasoning (ADR) — image + audio, 146 instances, resolve visual ambiguity via acoustic cues: 27.40%, the floor of the whole eval
  • Video-based Decision Reasoning (VDR) — prefix-video continuation, 100 instances, respond appropriately to observed dynamics: 56.00%, the ceiling
  • Audiovisual Integrated Reasoning (AVIR) — video + audio, 71 instances, fold auditory evidence or spoken constraints into continuation and editing: 47.89%

Scoring is not answer matching: three experts independently judge whether each output satisfies the intended semantic constraint, and multiple valid visual realizations are allowed. The overall rate is computed over all 517 instances, not averaged across scenarios.

Behind the 42%: generating well does not mean listening well

Two findings stand out. First, audio grounding is the shortest plank. The listed failure modes include associating a sound with the wrong visible source and confusing similar mechanical sounds. The framework's README states it plainly: visual plausibility is insufficient — a video can look convincing while failing the required spatial, temporal or audiovisual condition. Second, video tasks (56%) clearly beat audio tasks (27.4%); the team's own conclusion is that supporting multimodal inputs does not ensure reliable task completion, and effective multimodal integration remains an open problem.

The paper also pre-empts over-reading: the four scenarios differ in data, prompts and generation targets, so these rates describe task-level performance, not a controlled comparison of the intrinsic value of modalities; and generated outputs are behavioral evidence — a failure could originate in perception, evidence integration, or generation.

Two buckets of cold water, then one fair point

Cold water #1: the evaluation dataset, the results, and the auto-evaluation pipeline are all still unchecked TODOs in the repo (the team says it is building a DeepSeek-powered automated pipeline), so every number is a single-team self-report with no third-party replication yet. Cold water #2: only MiniMax-H3 was tested — "omni models in general are like this" is an extrapolation, not a finding.

The fair point: the question this eval raises is more valuable than its numbers. The public omni-generation race has concentrated on visible metrics — resolution, duration, visual quality — while the paper argues existing frameworks rarely exploit omni-modal inputs at all. It ranked #2 on Hugging Face's Daily Papers board for Sep 18 (92 upvotes), so the community is clearly hungry for this direction. For video-generation teams, audio-visual alignment may be the most underestimated weak spot.

So the next time you see a demo claiming "one model generates the whole soundscape," ask one more question: it generated — but did it understand what it heard?

References: paper arXiv:2609.18323 · framework GitHub: gulucaptain/MiniMax-H3-Reason