[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ambit-spatial-audio-scene-plan":3,"topics-all":38,"news-related-1d85eeac-c803-453e-99f2-cad5fa838d1e":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"1d85eeac-c803-453e-99f2-cad5fa838d1e","AMBIT:空间音频的编辑接口,是一份可执行的场景计划","港科大与 Noiz AI 等机构提出 AMBIT:在音频渲染前插入一份可执行的场景计划,用同一套 Transformer 骨干兼任自回归规划与整流流渲染,统一做一阶 Ambisonics 空间音频的生成与编辑。论文自测显示其规划可执行率 100%,编辑时未改动声源的方向 MAE 为 0.17,明显优于级联基线。","空间音频的难处不在把声音渲染得像,而在于让人听得出谁在哪儿。脚步声从身后传来、引擎声在右前方逐渐逼近,这些都要求声源的方向、距离和运动轨迹同时成立。过去的做法通常把生成和空间化拆成两段:先按文本生成音频,再用空间化算法把声音摆到指定位置。生成阶段管内容,空间化阶段管方位,链路短、组件成熟,但一做编辑就露馅——改一个声源,整张声场的几何关系都得重算,改动对象和被保留对象之间没有共享的表示。\n\n## 一份可执行的场景计划\n\nAMBIT 的做法是在音频前面插一层显式计划。作者把请求编译成 ScenePlan,里面写明房间类型、时长(1 到 648 帧)、声源清单、描述与转写、运动方式与起止方位(八个方位类别)、起止时间(五个相对阶段)、径向运动(自由、靠近或远离)。这些字段不是自由文本:整份计划由语法受限的贪心解码产出,编码词表共 4,096 个 token,包含 1,536 个文本片段、649 个帧值、1 度的角度分箱,以及 256 个覆盖 0.1 到 50 米的对数距离分箱。关键在于这份计划同时是生成的输入和编辑的接口:要改什么、保留什么,都写在同一份结构里。\n\n## 同一个骨干干两件事\n\n计划用自回归解码产出,声音用整流流渲染,两者共享同一套 Transformer 块:15 个共享块,宽度 1,024,16 个注意力头,连同接口与结构化头共 3.326 亿可训练参数。文本侧挂的是 Qwen3.5-0.8B 骨干,训练全程冻结,另外冻结的还有 Ambit CLAP 和 Ambit VAE;规划侧额外挂一个 rank 为 8 的 LoRA,只在自回归阶段生效。训练数据是 160 万条合成与实录 FOA 样本加 100 万组编辑对,VAE 另用约 100 万段 4 秒 FOA 片段预训练,机器是 8 张 L40S。评测侧用 8,000 条生成请求、5,000 组编辑对和 2,000 段 VAE 测试片段。\n\n## 空间信息不能在重建里被吃掉\n\nFOA 有四个通道,解码器一不留神就会抹平通道之间的相位关系。论文拿 DAC、EnCodec、WavTokenizer 做对照,自家 Ambit VAE 的全向加权 SI-SDR 是 6.39 dB,DAC 是 5.57 dB,EnCodec 只有 0.41 dB;方向误差 4.96 度,而对照里最好的 DAC 是 5.13 度。消融路径更能说明问题:只加空间协方差匹配,方向误差从 6.51 度压到 5.14 度,SI-SDR 从 5.63 涨到 6.08 dB;再叠倒谱项和 SI-SDR 项到 6.25 dB;最后靠分组 KL 配全向恢复做到 6.39 dB,强度一致性误差降到 0.04。\n\n## 真正的硬骨头是编辑\n\n生成数字漂亮,编辑数字才见真章。编辑内容质量六项(配对 CLAP、FD-CLAP、FAD、FD-PANNs、KL、LSD)上,Ambit 全面优于论文对比集里最强的基线 SwanWeave:配对 CLAP 0.84 对 0.68,LSD 1.27 对 1.65。更关键的是没被动过的部分有没有保住:未编辑声源的方向 MAE 是 0.17,而 Qwen3-Omni 加 DiT 的级联方案是 1.42、SwanWeave 是 0.90;绝对电平变化 1.65 对 20.94 和 8.40;角轨迹 RMSE 0.270 对 0.797 和 1.157。改一个声源,其余声源还能待在原地,靠的就是计划里显式写明了被保留的对象。\n\n## 自蒸馏:让计划来教渲染\n\n后训练阶段用了联合 Transfusion 在策略自蒸馏(OPSD):学生自己生成计划前缀,渲染也在学生计划实际走到的采样状态上做蒸馏,目标条件教师同时监督两种模式。对照持续 SFT、ReFL、GRPO 以及两个分支单独蒸馏,联合 OPSD 在报告的全部九项指标上都最好。相对 SFT,配对 CLAP 从 0.855 到 0.867,GCC 从 11.897 到 11.130,CRW 从 7.518 到 5.854。分支单独蒸馏的效果不齐:只在渲染侧蒸馏能压过 SFT 的每一项,而只在规划侧蒸馏相对 SFT 只在 LSD 和 FSAD 两项上有改善,其余反而变差。\n\n## 泼点冷水\n\n听感评测用耳机和固定双耳解码器,每个任务 400 个用例,合计 2,400 次系统输出呈现。Ambit 在空间遵循上拿 3.37 分,比论文对比集里最强的基线高 0.54 分,指令遵循高 0.29 分;编辑三项是 3.64、3.57、3.71,超过 Qwen3-Omni 加 DiT 的 3.51、3.26、3.44。但论文自己也留了几处短板:语音 UTMOS 只有 3.52,低于 Qwen3-TTS 的 4.18;音乐与音效上的 FD-PaSST、FD-OpenL3 仍低于 AudioX-Turbo;当前场景计划配置只支持约 15 秒;所有跑分都是论文自己的评测,社区尚无第三方复现。还有一处容易被误读:计划可执行率对比里,通用 Qwen3.5-0.8B 是 22.88%、2B 是 20.19%、4B 98.08%、9B 87.81%,自家规划器 100%,但论文明确说明对方只取第一次响应、不做修复或重试,自家走语法受限解码——这是两套解码设置下的比较,不是纯粹的模型规模对比。\n\n## 所以呢\n\nAMBIT 真正有意思的地方,是把空间音频当成一件有结构的工程来做:先写一份能被程序读懂、能被程序检查的计划,再照着渲染。这样做的好处不止空间精度——计划本身成了编辑接口,也成了可验证对象,后面接自动混音、虚拟演出、游戏运行时音频都能复用同一份表示。代价是链路更长、对计划语法的设计要求更高。论文全文见 [AMBIT: Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.34415)。如果音视频工具链里已经在攒\"结构化中间表示\"这条线,这份工作值得当作参照样本读一遍。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.34415","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"e676a5cf-1f24-472f-a765-86fa21a1bc3c","ai-model",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"395fe699-4ae2-4d87-a706-c38c2da6ba69","en","AMBIT: Spatial Audio Editing via Executable Scene Plans","AMBIT puts an executable scene plan in front of Ambisonics rendering: one Transformer backbone handles both planning and rectified-flow rendering.","Spatial audio is hard not because of how convincingly a sound renders, but because the listener has to work out who is where. Footsteps behind you, an engine creeping up in front and to the right — all of that demands source direction, distance and motion trajectory hold together. The usual pipeline splits the job in two: generate audio from text first, then push the result into position with a spatializer. One stage handles content, the other handles placement. The chain is short and the components are mature, but editing exposes the seam: change one source and the whole sound field's geometry has to be recomputed, because the object being changed and the objects being preserved share no representation.\n\n## An Executable Scene Plan\n\nAMBIT inserts an explicit plan in front of the audio. The request is compiled into a ScenePlan that names the room type, the duration (1 to 648 frames), the source inventory, descriptions and transcripts, motion type and start\u002Fend azimuth (eight compass categories), onset and offset (five relative time phases), and radial motion (free, approaching or receding). None of these fields are free text: the entire plan comes out of grammar-constrained greedy decoding, against a codec vocabulary of 4,096 tokens covering 1,536 text pieces, 649 frame values, 1-degree angular bins, and 256 logarithmic distance bins spanning 0.1 to 50 meters. The important part is that this plan serves as both the input to generation and the interface for editing — what to change and what to retain are written into the same structure.\n\n## One Backbone Doing Two Jobs\n\nThe plan is produced autoregressively and the sound is rendered with a rectified flow, and both share the same Transformer blocks: 15 shared blocks, width 1,024, 16 attention heads, totaling 332.6 million trainable parameters including the native interfaces and structured heads. The text side hangs off a Qwen3.5-0.8B backbone that stays frozen throughout training, as do Ambit CLAP and the Ambit VAE; the planning side adds a LoRA of rank 8, active only in autoregressive mode. Training data is 1.6 million synthetic and recorded FOA examples plus 1 million editing pairs, the VAE is pretrained on roughly 1 million four-second FOA clips, and the hardware is eight NVIDIA L40S GPUs. Evaluation uses 8,000 generation requests, 5,000 editing pairs, and 2,000 VAE test clips.\n\n## Spatial Information Must Survive Reconstruction\n\nFOA has four channels, and a decoder will flatten the phase relationships between them if it is not careful. Against DAC, EnCodec and WavTokenizer, the Ambit VAE reaches a WW-weighted SI-SDR of 6.39 dB, versus 5.57 dB for DAC and just 0.41 dB for EnCodec; its direction error is 4.96 degrees, where the best of the baselines, DAC, sits at 5.13 degrees. The ablation path makes the point even more clearly: adding spatial covariance matching alone drops direction error from 6.51 to 5.14 degrees and lifts SI-SDR from 5.63 to 6.08 dB; stacking a cepstral term and an SI-SDR term reaches 6.25 dB; and grouped KL with omnidirectional recovery finally lands on 6.39 dB with intensity coherence error down at 0.04.\n\n## Editing Is the Real Test\n\nThe generation numbers look good, but the editing numbers are where the truth shows. Across six editing content-quality metrics (paired CLAP, FD-CLAP, FAD, FD-PANNs, KL, LSD), Ambit beats the strongest baseline in the paper's comparison set, SwanWeave, across the board: paired CLAP 0.84 against 0.68, LSD 1.27 against 1.65. What matters more is whether the parts that should not have moved stayed put. For unedited sources, Ambit's direction MAE is 0.17, while the cascaded Qwen3-Omni plus DiT setup sits at 1.42 and SwanWeave at 0.90; absolute level change is 1.65 against 20.94 and 8.40; angular trajectory RMSE is 0.270 against 0.797 and 1.157. Change one source and the rest still hold their positions — that works because the plan explicitly states which sources are retained.\n\n## Self-Distillation: Let the Plan Teach the Renderer\n\nPost-training uses joint Transfusion on-policy self-distillation (OPSD): the student generates its own plan prefixes, and rendering is distilled at sampling states the student's plan actually reaches, with target-conditioned teachers supervising both modes. Against continued SFT, ReFL, GRPO, and both single-branch supervision variants, joint OPSD comes out best on all nine reported metrics. Relative to SFT, paired CLAP moves from 0.855 to 0.867, GCC from 11.897 to 11.130, and CRW from 7.518 to 5.854. Single-branch distillation is uneven: distilling the renderer alone beats SFT on every metric, while distilling the planner alone improves only LSD and FSAD over SFT, with the rest regressing.\n\n## Some Cold Water\n\nThe listening study used headphones and a fixed binaural decoder, with 400 cases per task and 2,400 system-output presentations in total. Ambit scores 3.37 on spatial adherence, 0.54 points above the strongest baseline in the paper's comparison set, and leads on instruction adherence by 0.29; its three editing measures are 3.64, 3.57 and 3.71, ahead of Qwen3-Omni plus DiT at 3.51, 3.26 and 3.44. But the paper leaves a few soft spots of its own: speech UTMOS is only 3.52, below Qwen3-TTS at 4.18; FD-PaSST and FD-OpenL3 on music and sound still trail AudioX-Turbo; the current ScenePlan configuration only supports scenes of roughly 15 seconds; and every number above is the authors' own evaluation, with no third-party replication yet. One comparison is also easy to misread: in the planner executability table, general-purpose Qwen3.5-0.8B scores 22.88%, 2B scores 20.19%, 4B scores 98.08% and 9B scores 87.81%, against 100% for the in-house planner — but the paper states plainly that the Qwen side takes its first response with no repair or retries while the in-house planner uses grammar-constrained decoding. That is a comparison across two decoding setups, not a clean test of model scale.\n\n## So What\n\nWhat makes AMBIT interesting is treating spatial audio as structured engineering: write a plan that programs can both read and check, then render against it. The payoff goes beyond spatial accuracy — the plan itself becomes the editing interface and a verifiable object, so automated mixing, virtual production and game runtime audio can all reuse the same representation. The cost is a longer chain and a higher bar for designing the plan grammar. The full paper is [AMBIT: Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.34415). If your audio\u002Fvideo toolchain is already collecting \"structured intermediate representation\" ideas, this one is worth reading as a reference sample.","ambit-spatial-audio-scene-plan","2026-10-27T09:00:00Z","2026-10-27T09:23:38.021872Z","2026-10-27T09:23:38.021881Z",true,"agent",120,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"e4ac848d-d24a-465f-bece-37e090da86de","全双工语音模型长对话必崩:KAIST 揪出初始化锚点被挤走","full-duplex-speech-collapse-fd-lhb-kv-cache","2026-10-27T03:20:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"db8f2b94-8565-49fe-9fd8-08310f441161","HiSTEVE 1.0:像人一样玩我的世界,只靠屏幕和键鼠","histeve-hierarchical-vla-minecraft-agent","2026-10-26T15:13:44+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"d34bdd08-5fa9-4ce0-954d-17406b896756","牛津102页综述:视频世界模型的四级证据阶梯","video-world-model-evidence-ladder-survey","2026-10-26T11:13:04+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"6ca2aad2-fc68-46ab-8120-3eada9c187e2","模型自己操控相机建图:最强 VLM 平均 IoU 只有四成","spatialscout-vlm-active-3d-mapping-benchmark","2026-10-23T03:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"bbedb317-c221-43e3-b9a4-61798ed17b55","Voxtral 3:Mistral 语音智能体全栈,一轮对话不到一秒","voxtral-3-voice-agent-sdk","2026-10-21T23:11:24+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"6b2cf1d1-7131-4885-a049-730b82846961","RISE-Agent 拆开视觉编辑推理盲点:58 个模型里,最强闭源只 56.6%","rise-agent-visual-editing-benchmark","2026-10-13T11:00:00+00:00"]