Video diffusion models can already generate photorealistic images, but they still have many holes in physical realism: objects floating, collisions with no bounce, soft matter deforming unrealistically. The PhyCo paper accepted at CVPR 2026 proposes a feasible path to make generated videos conform to real-world physical laws.
Core problem: diffusion models are good at "looking real," not "moving real."
Existing methods either rely on explicit physics simulators (like PhysGen, PhysDreamer), requiring 3D geometry reconstruction or preset materials, with high inference-time compute cost and poor generalization; or rely on implicit guidance (Force Prompting, VLIPP), improving semantic consistency but unable to do continuous, controllable, precise adjustment of physical properties.
PhyCo systematically addresses this problem at three levels: data, architecture, and training.
First, a large-scale physics-simulation dataset: over 100,000 photorealistic simulation video segments, systematically varying friction coefficient, elasticity recovery coefficient, deformation degree, force magnitude, covering multiple scenarios.
Second, ControlNet-based physics-supervised fine-tuning: using pixel-aligned physical-property maps as conditions, applying physics-supervised fine-tuning to pretrained diffusion models, turning physical properties from implicit to explicit "knobs," continuously adjustable.
Third, VLM-guided reward optimization: using a fine-tuned vision-language model to score generated videos, receiving differentiable feedback signals, achieving end-to-end physics consistency reinforcement, with zero extra overhead at inference.
On the Physics-IQ benchmark, PhyCo significantly outperforms strong baselines; human evaluation also confirms its clearer, more accurate control over physical properties. After training on synthetic data, the model can generalize to realistic scenarios — something prior methods struggled to do.
The significance of this work goes beyond "making AI-generated bouncy balls more realistic." It points to a deeper issue: current video generation models are essentially fitting statistical patterns of pixel distributions, not understanding causal physical mechanisms. PhyCo proves that by introducing physics priors and data design, you can embed understanding of real-world laws in large-scale generative models. Of course, the sim-to-real Domain Gap still exists, and some scenarios in the paper do show some stylized rendering traces. But as the first work to simultaneously achieve "continuous physical control" and "zero inference overhead," PhyCo provides a good foundation for the next stage of physics-aware video generation.