Making a robot "think one step, act one step" sounds trivial — the hard part is imagination. Today's vision-language models can read a scene, but they cannot simulate how the world will change after an action. RynnUnify, released on arXiv on October 23 by Alibaba's DAMO Academy, packs understanding and imagination into a single model: a 16B unified embodied model, with weights already open-sourced.

One Architecture, Two Experts

RynnUnify uses a Mixture-of-Transformers design: an 8B autoregressive Reasoner handles reasoning, spatial grounding, and subtask planning, while an 8B rectified-flow video Generator renders "what happens next" into video segments. The two are connected through joint attention, and the Generator conditions on scene memory and recent motion. The whole model is trained on top of NVIDIA Cosmos3 Nano; the README acknowledges cosmos-framework, RynnBrain, and Relax as upstream projects.

Imagine First, Then Act

The core mechanism is interleaved visual planning: the Reasoner proposes the next subtask, the Generator produces a video of its predicted outcome, and the model plans the next move from that imagined footage. Cross-chunk scene memory, self-forcing training, and adaptive durations keep long-horizon plans coherent. The more interesting piece is BCPO (Branch-Conditional Policy Optimization): for each routable question, it samples four direct and four generation-assisted trajectories; within each branch, GSPO improves reasoning, while reward differences between branches teach the model when imagination is worth the cost. During training the Generator stays frozen; only the agent is updated.

The Official Numbers

On the team's own Rynn-VP benchmark, RynnUnify scores 54.80 overall, 9.75 points above Cosmos 3 Nano's 45.05. On Rynn-GEU, enabling the Agentic mode yields 50.0 versus 44.0 for direct reasoning. Across ten embodied-understanding benchmarks it averages 57.9, above Cosmos 3 Nano (53.2), RxBrain (44.9), and BAGEL (28.0) — though the team's own understanding-only RynnBrain-8B reaches 61.2 in the same official table, so unification has a visible cost, and a GPT-5.6 Terra + MiniMax-H3 composed system scores higher at 65.71 on Rynn-VP. On the robot side, the WAM extension hits 51.5% success on RoboCasa365 Selected-20 (200 episodes), ahead of CMA's 47.0 and Xiaomi-Robotics-1's 46.0; on three real-robot tasks on the Astribot S1 it averages 46.7%, against 26.7% for GR00T N1.7 and 25.0% for pi-0.5.

Open Source, and a Cold Shower

Three checkpoints — RynnUnify-Base, Agentic-Reasoner, and WAM-RoboCasa — are on Hugging Face and ModelScope, with Apache-2.0 code. The cold water: Rynn-VP and Rynn-GEU are the team's own benchmarks, every comparison above comes from the official evaluation, and no third-party reproduction has appeared yet; the real-robot evaluation covers only three tasks. Still, the idea of treating a world model as an on-demand tool during reasoning — and using RL to learn when to call it — feels closer to what a "brain" means than merely scaling world-model parameters. Knowing when to imagine is itself a form of intelligence.

References: arXiv:2610.32965; GitHub; Project page