A study submitted to arXiv on August 20 (arXiv:2608.20335) connects "a casually shot video" directly to "a 4D human you can orbit freely": the 4DAnyone framework takes an uncalibrated monocular video as input, first generates tens of multiview-consistent target videos, and then lifts them into a 4D Gaussian Splatting (4DGS) human model. The paper is a collaboration between five institutions - Zhejiang University, Ant Group, Robbyant, HKUST, and CUHK - with the project page listing SIGGRAPH Asia 2026 as the venue; both code and model weights are already public.

Filling In the Virtual Camera Array with a Generative Model

As the project page describes it, photorealistic 4D human reconstruction normally depends on a calibrated array of synchronised cameras, which severely limits real-world use. 4DAnyone flips the direction: what reconstruction needs is precisely "the tens of videos that array would have recorded", so let a video diffusion model generate them, and train 4DGS on the results. The project page's pitch is one line: single video in, 4D human out - no rig, no calibration, no tripod.

Why Consistency Breaks at Tens of Views

Scaling an off-the-shelf camera-controlled video diffusion model to this size breaks multiview consistency. The paper traces the failure to a "bounded-attention-context" problem: once the number of target views exceeds the capacity of a single DiT forward pass, generation must be split into groups, which exposes two coupled bottlenecks. On the reference side, conditioning on all previously generated views makes the context grow as O(N), weakening cross-view appearance guidance. On the target side, disjoint groups cannot directly exchange information, causing global structural drift.

The two corresponding designs are decidedly engineering-flavored:

  • Reference Context Packing (RCP) compresses the growing reference views into a fixed-length mixed-resolution context, dropping reference-side complexity from O(N) to O(1);
  • Target Context Routing (TCR) rotates the target-view groupings during denoising - sharing context across groups at high-noise steps and stabilizing details at low-noise steps.

There is one more key trade-off in the conditioning signal: a 3D skeleton supplies sparse-but-accurate geometric cues, replacing fragile dense depth and noisy camera parameters. The authors credit this for the model's robust in-the-wild generalization.

Game-Engine Data, Two Benchmarks

For training data, the team built the MVGameHuman dataset with an in-house game engine, mixing it with light-stage data and in-the-wild videos. Evaluation runs on DNA-Rendering and DyMVHumans: the paper reports it outperforms prior methods on both novel-view video quality and downstream 4DGS reconstruction, with robust in-the-wild generalization; the final 4DGS model is trained with FreeTimeGS.

Open Source and Community Traction

Code is hosted on GitHub (ant-research/4DAnyone) and weights on Hugging Face (AntResearch/4DAnyone). On August 21 the paper reached #3 on Hugging Face Daily Papers with 64 upvotes. For workflows built on free-viewpoint volumetric video and digital humans, this is a pipeline you can actually run, not just a method that lives in a paper.

Seen from a wider angle, 4DAnyone represents a new class of solutions: where reconstruction (4DGS) and generation (video diffusion) meet at the interface of "multiview consistency", the bottleneck is no longer only on the reconstruction side but in context management on the generation side - RCP and TCR are, in essence, budget management for the DiT attention context. The same lesson transfers to other tasks that need consistency at scale: when you run out of context, compress it - do not just keep stacking it.