On September 10, the Vidu team posted the S2 technical report on arXiv (2609.11638); five days later it climbed to #1 on Hugging Face Daily Papers with 393 upvotes. S1 only shipped in July, and S2 is already here — the density of this upgrade deserves a closer look.
One upgrade, three capabilities
S2 is no longer a single model but two: S2-Avatar handles real-time interactive digital characters, while S2-Editing handles real-time video editing. On the avatar side, resolution moves from S1's 540p up to 720p at 25-42 FPS. More important is "dynamic references": drop a new reference image at any moment mid-stream to change outfits, props, or scenes without restarting. Instruction following also got stronger, expanding beyond talking heads to large body motions such as dancing. S2-Editing transforms an incoming video stream in real time — style transfer, virtual try-on, character replacement, background replacement — while preserving the source motion and timing. One more forward-leaning step: generated or edited streams can be converted in real time into synchronized left- and right-eye views for stereoscopic video, streamed straight to VR headsets.
Fighting streaming drift with Self-Replay Forcing
The chronic pain of streaming video generation is error accumulation: each segment is conditioned on the previous segment's own output, so small errors snowball. S2's answer is Self-Replay Forcing (SRF) — re-noising and replaying the model's self-generated trajectories through a gradient-enabled causal pass, teaching the model to "catch its own output" rather than drift with it, and preventing errors from accumulating across streaming segments. This is a head-on attack on streaming stability, not hand-waving.
The inference stack is the hidden protagonist
720p at 25-42 FPS is not carried by the model alone. The official GitHub repo (shengshu-ai/Vidu-S) is explicit: a TurboDiffusion plus TurboServe serving stack combines efficient attention, low-bit GEMM, kernel and launch optimizations, and multi-GPU pipelining to squeeze real-time inference onto low-cost GPUs. The authors report that S2 outperforms all baselines across the five public benchmarks in their evaluation — note this is a self-reported claim, independent reproduction has not appeared yet, so treat it with care.
So what
From S1 to S2 the roadmap is now clear: real-time video generation is graduating from demo to infrastructure. Avatar targets digital-human livestreaming and companionship, Editing targets virtual try-on and film previz, spatial video targets VR — each lands on a concrete application. The real open question is cost: low-bit GEMM and pipelining optimizations decide whether this stack can run long-term on cheap GPUs. A playable demo is live at vidu.com/vidu-stream with API docs published alongside; if you want to verify, running it yourself beats reading benchmarks.