Spatial audio is hard not because of how convincingly a sound renders, but because the listener has to work out who is where. Footsteps behind you, an engine creeping up in front and to the right — all of that demands source direction, distance and motion trajectory hold together. The usual pipeline splits the job in two: generate audio from text first, then push the result into position with a spatializer. One stage handles content, the other handles placement. The chain is short and the components are mature, but editing exposes the seam: change one source and the whole sound field's geometry has to be recomputed, because the object being changed and the objects being preserved share no representation.

An Executable Scene Plan

AMBIT inserts an explicit plan in front of the audio. The request is compiled into a ScenePlan that names the room type, the duration (1 to 648 frames), the source inventory, descriptions and transcripts, motion type and start/end azimuth (eight compass categories), onset and offset (five relative time phases), and radial motion (free, approaching or receding). None of these fields are free text: the entire plan comes out of grammar-constrained greedy decoding, against a codec vocabulary of 4,096 tokens covering 1,536 text pieces, 649 frame values, 1-degree angular bins, and 256 logarithmic distance bins spanning 0.1 to 50 meters. The important part is that this plan serves as both the input to generation and the interface for editing — what to change and what to retain are written into the same structure.

One Backbone Doing Two Jobs

The plan is produced autoregressively and the sound is rendered with a rectified flow, and both share the same Transformer blocks: 15 shared blocks, width 1,024, 16 attention heads, totaling 332.6 million trainable parameters including the native interfaces and structured heads. The text side hangs off a Qwen3.5-0.8B backbone that stays frozen throughout training, as do Ambit CLAP and the Ambit VAE; the planning side adds a LoRA of rank 8, active only in autoregressive mode. Training data is 1.6 million synthetic and recorded FOA examples plus 1 million editing pairs, the VAE is pretrained on roughly 1 million four-second FOA clips, and the hardware is eight NVIDIA L40S GPUs. Evaluation uses 8,000 generation requests, 5,000 editing pairs, and 2,000 VAE test clips.

Spatial Information Must Survive Reconstruction

FOA has four channels, and a decoder will flatten the phase relationships between them if it is not careful. Against DAC, EnCodec and WavTokenizer, the Ambit VAE reaches a WW-weighted SI-SDR of 6.39 dB, versus 5.57 dB for DAC and just 0.41 dB for EnCodec; its direction error is 4.96 degrees, where the best of the baselines, DAC, sits at 5.13 degrees. The ablation path makes the point even more clearly: adding spatial covariance matching alone drops direction error from 6.51 to 5.14 degrees and lifts SI-SDR from 5.63 to 6.08 dB; stacking a cepstral term and an SI-SDR term reaches 6.25 dB; and grouped KL with omnidirectional recovery finally lands on 6.39 dB with intensity coherence error down at 0.04.

Editing Is the Real Test

The generation numbers look good, but the editing numbers are where the truth shows. Across six editing content-quality metrics (paired CLAP, FD-CLAP, FAD, FD-PANNs, KL, LSD), Ambit beats the strongest baseline in the paper's comparison set, SwanWeave, across the board: paired CLAP 0.84 against 0.68, LSD 1.27 against 1.65. What matters more is whether the parts that should not have moved stayed put. For unedited sources, Ambit's direction MAE is 0.17, while the cascaded Qwen3-Omni plus DiT setup sits at 1.42 and SwanWeave at 0.90; absolute level change is 1.65 against 20.94 and 8.40; angular trajectory RMSE is 0.270 against 0.797 and 1.157. Change one source and the rest still hold their positions — that works because the plan explicitly states which sources are retained.

Self-Distillation: Let the Plan Teach the Renderer

Post-training uses joint Transfusion on-policy self-distillation (OPSD): the student generates its own plan prefixes, and rendering is distilled at sampling states the student's plan actually reaches, with target-conditioned teachers supervising both modes. Against continued SFT, ReFL, GRPO, and both single-branch supervision variants, joint OPSD comes out best on all nine reported metrics. Relative to SFT, paired CLAP moves from 0.855 to 0.867, GCC from 11.897 to 11.130, and CRW from 7.518 to 5.854. Single-branch distillation is uneven: distilling the renderer alone beats SFT on every metric, while distilling the planner alone improves only LSD and FSAD over SFT, with the rest regressing.

Some Cold Water

The listening study used headphones and a fixed binaural decoder, with 400 cases per task and 2,400 system-output presentations in total. Ambit scores 3.37 on spatial adherence, 0.54 points above the strongest baseline in the paper's comparison set, and leads on instruction adherence by 0.29; its three editing measures are 3.64, 3.57 and 3.71, ahead of Qwen3-Omni plus DiT at 3.51, 3.26 and 3.44. But the paper leaves a few soft spots of its own: speech UTMOS is only 3.52, below Qwen3-TTS at 4.18; FD-PaSST and FD-OpenL3 on music and sound still trail AudioX-Turbo; the current ScenePlan configuration only supports scenes of roughly 15 seconds; and every number above is the authors' own evaluation, with no third-party replication yet. One comparison is also easy to misread: in the planner executability table, general-purpose Qwen3.5-0.8B scores 22.88%, 2B scores 20.19%, 4B scores 98.08% and 9B scores 87.81%, against 100% for the in-house planner — but the paper states plainly that the Qwen side takes its first response with no repair or retries while the in-house planner uses grammar-constrained decoding. That is a comparison across two decoding setups, not a clean test of model scale.

So What

What makes AMBIT interesting is treating spatial audio as structured engineering: write a plan that programs can both read and check, then render against it. The payoff goes beyond spatial accuracy — the plan itself becomes the editing interface and a verifiable object, so automated mixing, virtual production and game runtime audio can all reuse the same representation. The cost is a longer chain and a higher bar for designing the plan grammar. The full paper is AMBIT: Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion. If your audio/video toolchain is already collecting "structured intermediate representation" ideas, this one is worth reading as a reference sample.