[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-l3g-world-gaussian-memory-realtime":3,"topics-all":38,"news-related-85b85139-0d39-464a-99cf-415fd498e9d2":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"85b85139-0d39-464a-99cf-415fd498e9d2","L3G-World:高斯记忆让世界模型9 FPS实时生长","世界模型的老毛病:走得越远,记忆越涨,直到 OOM。KAIST 牵头的 L3G-World 把场景记忆装进潜空间 3D 高斯,记忆量随场景范围而非视频长度增长——单张 H100 峰值 7.4 GiB、9.02 FPS 实时生成,481 帧回环 PSNR\u002FLPIPS 最优,蒸馏版一致性回落。","交互式世界模型有一个不常被聊起的死穴:你在生成的世界里走得越远,机器就越吃力。主流做法要么把历史帧或 KV 缓存当上下文一路拖着走,要么把生成过的画面重建成点云、高斯这样的显式 3D 记忆——但两条路的成本都随视频流长度线性上涨,走到一定距离,要么显存爆掉,要么帧率低到没法交互。KAIST 牵头的团队 10 月 29 日挂上 arXiv 的论文里,时序上下文路线的 WorldKV 在 400 帧生成时直接 OOM,Yume 1.5 也要 66.7 GiB 才换来 1.42 FPS。\n\n## 记忆随场景涨,不随时间涨\n\n论文的解法叫 L3G(Latent 3D Gaussian Memory):一个记忆更新器 L3G-Mem 和一个视频扩散 Transformer L3G-Gen 交替自回归地工作;训练上,L3G-Mem 用 DL3DV,L3G-Gen 用 DL3DV 加 RealEstate10K,实时版 L3G-World Fast 靠 AnyFlow 少步蒸馏得来。关键设计有两个。第一,记忆只在尚未覆盖的区域生成新高斯,对已有高斯做残差式修正——记忆不是只进不出的缓冲区,而是持续被改写的持久场景表示,尺寸随场景覆盖范围增长(复杂度 O(S)),而不是随生成帧数增长(O(T·R))。第二,每个高斯直接存一份 VAE 潜特征,记忆可以径直渲染进视频生成器的潜空间,省掉「渲出 RGB 再过一遍 VAE 编码」的往返。\n\n## 直接在潜空间渲染,快两个数量级\n\n第二个设计是速度的关键。传统显式 3D 记忆要给潜空间视频生成器当条件,必须先渲 RGB 再重新 VAE 编码,这个往返让实时生成无从谈起。消融实验把往返的成本量了出来:去掉潜特征直存、退回 RGB 渲染加 VAE 编码后,VAE 空间渲染吞吐从 943.6 FPS 崩到 2.97 FPS。论文报告的对比数字是:与各指标上最高效的既有 3D 记忆基线相比,3D 图元少 38.6 倍、记忆更新快 3.2 倍、记忆渲染快 359 倍。\n\n## 单卡 H100:7.4 GiB,9.02 FPS\n\n绝对数字更直观。400 帧生成、单张 H100 上,L3G-World 峰值显存 7.4 GiB、整体 9.02 FPS,是对比表里唯一标成「实时」的方法;同为高斯路线的 NeoVerse 要 36.8 GiB 换 1.99 FPS,走优化路线的 ArtiFixer 只要 11.7 GiB 却慢到 0.10 FPS。在 DL3DV 与 RealEstate10K 两组数据集、8 个基线的对比里,论文构造 481 帧的「走出去再原路折返」回环轨迹测长时程自一致性——同一条相机路径正着走一遍、倒着走一遍,比对去程与回程在同一相机位姿下的画面差异,L3G-World 的 PSNR(16.00\u002F18.36)与 LPIPS 均为最优,3D 图元数量比 NeoVerse 少 266 到 417 倍;与点云路线里效率最好的 Lyra 2.0 相比,记忆重建快 8.9 到 9.5 倍、显存占用低 7.8 到 10.4 倍。\n\n## 泼点冷水\n\n快是有代价的。为实时而蒸馏的 L3G-World Fast 版,自一致性 PSNR 从 16.00 回落到 14.59;生成分辨率 480×832,放在 2026 年不算出众;整套系统搭建在 Cosmos-Predict2.5、Depth Anything 3 等既有组件之上,训练用 4 张 A100,评测场景是单张图片加相机轨迹,离「开放世界游戏」还很远。\n\n但对这条赛道,论文留下的那个问题会比论文本身活得更久:长时序生成系统的记忆,到底该按时间增长,还是按空间增长?答案决定你能走多远。\n\n参考:[arXiv:2610.40614](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40614) · [项目页](https:\u002F\u002Fcvlab-kaist.github.io\u002FL3G-World\u002F)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40614","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":22,"name":23,"slug":23,"description":14,"color":14},"ebe5dcd1-46b1-4298-b8c2-8e0e2f456e56","video-generation",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"b43a9b3b-11ef-4789-bb65-95ad93a449e5","en","L3G-World: Gaussian Memory Makes World Generation Real-Time","KAIST's L3G-World scales world-model memory with scene extent, not stream length: 7.4 GiB, 9.02 FPS real-time on H100, best 481-frame loop closure.","Interactive world models have a failure mode that rarely makes headlines: the longer you walk through a generated world, the harder the machine has to work. Mainstream approaches either drag history frames or KV caches along as temporal context, or rebuild everything you generated into an explicit 3D memory such as point clouds or Gaussians. Both paths scale with the length of the generated stream. Past a certain distance you either run out of memory or sink below interactive frame rates. In the paper a KAIST-led team posted to arXiv on October 29, the temporal-context method WorldKV goes out-of-memory at 400 frames of generation, and Yume 1.5 needs 66.7 GiB to deliver 1.42 FPS.\n\n## Memory that scales with the scene, not the stream\n\nThe paper's answer is L3G, a Latent 3D Gaussian Memory: a memory updater (L3G-Mem) and a video diffusion transformer (L3G-Gen) alternate autoregressively; training uses DL3DV for L3G-Mem and DL3DV plus RealEstate10K for L3G-Gen, and the real-time L3G-World Fast comes from AnyFlow few-step distillation. Two design decisions carry the weight. First, memory is updated selectively: new Gaussians are spawned only in regions not yet covered, while existing ones receive residual refinement. Memory is not a buffer that only accumulates — it is a persistent scene representation that keeps being rewritten, so its size grows with the spatial extent of the scene (complexity O(S)) rather than with the number of generated frames (O(T·R)). Second, every Gaussian stores a VAE latent feature, so the memory renders directly into the video generator's latent space, skipping the render-to-RGB-then-re-encode round trip.\n\n## Rendering in latent space, two orders of magnitude faster\n\nThe second decision is where the speed comes from. Conditioning a latent video generator on an explicit 3D memory normally requires rendering RGB and re-encoding through the VAE, a round trip that rules out real time. The ablation quantifies the cost: remove direct latent storage, fall back to RGB rendering plus VAE encoding, and VAE-space rendering throughput collapses from 943.6 FPS to 2.97 FPS. Against the most efficient prior 3D-memory baseline for each metric, the paper reports 38.6x fewer 3D primitives, 3.2x faster memory updates, and 359x faster memory rendering.\n\n## One H100: 7.4 GiB, 9.02 FPS\n\nThe absolute numbers are just as telling. For 400-frame generation on a single H100, L3G-World peaks at 7.4 GiB and runs at 9.02 FPS overall — the only method in the comparison marked \"real-time\". NeoVerse, also Gaussian-based, needs 36.8 GiB for 1.99 FPS; the optimization-based ArtiFixer fits in 11.7 GiB but runs at 0.10 FPS. Across the DL3DV and RealEstate10K datasets and 8 baselines, the authors build 481-frame loop-closure trajectories — the same camera path walked forward, then retraced backwards — and compare frames at matching poses (PSNR\u002FSSIM\u002FLPIPS) to measure long-horizon self-consistency. L3G-World posts the best PSNR (16.00\u002F18.36) and LPIPS, with 266 to 417x fewer 3D primitives than NeoVerse, and 8.9 to 9.5x faster reconstruction on 7.8 to 10.4x less memory than the best point-cloud baseline, Lyra 2.0.\n\n## Some cold water\n\nSpeed has a price. The few-step-distilled L3G-World Fast gives back some self-consistency, with PSNR falling from 16.00 to 14.59 on DL3DV. Generation resolution is 480×832, unremarkable for 2026. The system is assembled from existing components — Cosmos-Predict2.5, Depth Anything 3 — trained on 4 A100 GPUs, and the evaluated setting is a single input image plus a camera trajectory, still far from an open-world game.\n\nFor this track, though, the question the paper raises will outlive the paper itself: should the memory of a long-horizon generative system grow with time, or with space? The answer decides how far you can walk.\n\nReferences: [arXiv:2610.40614](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40614) · [Project page](https:\u002F\u002Fcvlab-kaist.github.io\u002FL3G-World\u002F)","l3g-world-gaussian-memory-realtime","2026-10-30T21:12:08Z","2026-10-30T21:12:11.792472Z","2026-10-30T21:12:11.792484Z",true,"agent",356,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"0e44f256-e66e-495c-82e3-aae4dd5e2374","LiveEdit 把扩散视频编辑推到 12.66 FPS：清华让 AR 实时编辑走出 PPT","liveedit-ar-video-editing","2026-07-01T06:15:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"703ab8f9-4f6c-46fd-8b3c-f53387619cad","更稀疏、更清晰:视频扩散注意力砍掉九成","ab-attention-block-sparse-video-diffusion","2026-10-30T15:15:10+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"b860aced-d186-4d56-b751-53d5ca1ed658","OmniShowcase:4张参考图,主体转360度不变形","omnishowcase-360-subject-video","2026-10-29T23:06:48+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4349edac-997e-4980-95a0-58054a28a9dd","视频生成投机去噪:1.7 步顶 4 步,提速 1.5 倍","speculative-forcing-video-denoising","2026-10-16T19:11:48+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"0b0af156-7b5c-47d8-bfc2-de8cc0b22ce2","混元开源Prism:稀疏注意力让2K音视频训练提速2.5倍","prism-2k-video-audio-sparse-attention","2026-10-13T23:13:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"24d56562-4886-4169-8aa9-773f5fa34e2f","GRACE 把视频潜空间压 8 倍:提速 11 倍,跑分反超","grace-latent-compression-video-diffusion","2026-10-08T15:10:59+00:00"]