Most video generators are mute: the visuals come out first, and sound is either missing entirely or bolted on afterwards by a separate model. On August 31, the AMAP-ML team at Amap (publishing as the DreamX Team) posted the DreamX-Creator 1.0 technical report on arXiv, taking the opposite route — a compact 7B-parameter generator that jointly denoises audio and video inside one model.

Straight to the top of HF Daily Papers

Submitted to Hugging Face Daily Papers the next day, the paper (arXiv:2608.31106) had collected 81 upvotes as of writing, ranking first on the day's board. The companion GitHub repository (AMAP-ML/DreamX-Creator) sits at 82 stars under an Apache 2.0 license. The team lists ten authors, with Jiashu Zhu as first author and Xiangxiang Chu among the names.

The attention is not surprising. Native joint audio-video generation remains one of the open problems in video generation: the abstract points out that recent video generators "often omit audio or synthesize it in a separate stage," which limits reciprocal modeling between visual dynamics and acoustic events. DreamX-Creator's answer is to put both streams in one network.

Architecture: separate early, gated late

Conditioned on a first frame and a text prompt, the network processes the audio and video streams independently through the first half, then couples them in the latter half via Gated Cross-Modal Attention — every cross-modal attention head's output is modulated by token-wise and head-wise gates, which the authors say yields bidirectional audio-video interaction.

The training recipe has four components:

  • A unified Audio-Video Data System that constructs temporally coherent clips, produces structured multimodal annotations, and organizes clips into capability-oriented pools;
  • Progressive Joint Training: two audio-video pre-training stages followed by high-quality finetuning;
  • Audio-Video Reinforcement Learning with modality-aware multimodal feedback, routing video, audio, and cross-modal signals to the corresponding streams — effectively moving RL post-training from the text-only domain into audio-video;
  • Autoregressive 1-Step 2K Refinement: a bidirectional multi-step teacher is adapted into an autoregressive multi-step refiner, then distilled into a student that needs a single denoising evaluation per temporal chunk.

Per the official claim, overall performance is "competitive with state-of-the-art open-source systems."

Standing on Wan and MOVA

The README acknowledgements name two foundations: the Wan team (Wan2.2) and the OpenMOSS team (MOVA). In other words, the premise of "a compact 7B doing native audio-video" is that the open-source video-generation stack has matured into infrastructure — newcomers don't have to rebuild the base, only push on the incremental problems of joint modeling and RL post-training.

Weights not out yet — read the paper first

One cold shower: the GitHub roadmap shows two items checked — repository initialization and the 1.0 technical report — while the third, "release validated model weights, inference code, configurations, and evaluation tools," remains unchecked. So the current honest state of "democratizing" is: license settled (Apache 2.0), recipe published, weights on the way. Developers hoping to run it locally can only study the report for now.

So what

Two things make DreamX-Creator worth tracking. First, it pulls joint audio-video generation down from flagship-scale exclusivity to 7B, with an explicit commitment to release the compact generator and the 2K refiner. Second, its delivery cadence is the industry norm in miniature — paper first, weights later, with a time gap between the open-source promise and the runnable artifact. The sensible move now: read the report, and when the weights land, verify every line of the official claim yourself.

References: arXiv:2608.31106; github.com/AMAP-ML/DreamX-Creator