The next narrative in video generation is the "world model": not just rendering frames, but letting users step inside, explore, and interact. A real world has a bottom line, though—turn around and look back, and the room you just left should still be there. Orbia, a benchmark released on arXiv by a team including Shuicheng Yan and Mike Zheng Shou, audits exactly that: do generated videos form a consistent 3D world?

720 explore-and-revisit cases

Orbia's idea is simple: have a model generate an interactive exploration, then return to the start and look again. The benchmark ships 720 instances following six exploration-and-revisit protocols; short trajectories run about 22 seconds with one revisit, while minute-long extensions include three revisits to stress long-horizon memory. The data pipeline mixes public datasets, self-captured scenes, and UE simulation: among short cases, 120 indoor and 360 outdoor, 300 from real-world observations and 180 from simulation, standardized at 720p and 16 FPS. Evaluation spans three dimensions: input preservation, generated-content persistence, and 3D self-consistency (cross-view depth, feature correspondence, and held-out Gaussian rendering).

27 models, one harsh scoreboard

Via native-interface adapters, the team evaluated 27 models spanning text-, action-, and camera-conditioned interfaces. On control, world models average 57.29 versus 32.48 for text/image-to-video systems—a 24.80-point gap; the best TI2V model, Seedance2.5, reaches 43.23, still far below Lyra2's 81.64. Category champions: Genie3 leads the temporal control score, Lyra2 holds the highest mean control level, EVOKE tops world-model temporal quality, SolarWM-H3 takes geometric stability and the overall tier, and HY-WorldPlay has the highest mean geometry level; within the TI2V group, Seedance2.5 leads overall through stronger control while Cosmos3-Super leads quality and geometry.

Even Genie3 turns side-steps into rotations

Per-case inspection is more informative than totals: in several inspected Genie3 cases, lateral requests produce turns instead of translation—confusion between moving and turning that shows up in its lower Look Accuracy. Looking like a world and working like a map are separated by a structural-reliability gap: visual persuasiveness is not cross-view consistency. For anyone using world models as interactive training grounds, revisit consistency is a hard prerequisite—a simulator that quietly rewrites its own history teaches agents to tolerate contradiction.

The benchmark, code, and dataset are all open: project page at orbia-bench.github.io, code on GitHub (worldbench/Orbia), dataset on Hugging Face. What this audit gives the field is less a leaderboard than a shared ruler. Next time someone pitches an "interactive world," ask one question: look back—does it still hold?

Reference: arxiv.org/abs/2610.20693