[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-nesformer-qwen-nes-cpu-weights":3,"topics-all":38,"news-related-eb55e1cf-15c1-487f-845c-e81f87f11cfa":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"eb55e1cf-15c1-487f-845c-e81f87f11cfa","红白机 CPU 编进 Qwen 权重:没训练,纯编译","开发者 Drew Breunig 开源 nesformer:用编译器把红白机的 6502 CPU、2KB 内存和卡带 ROM 直接写进 Qwen3.5 标准权重,零训练。模型逐 token 预测 CPU 指令流,466.6M 参数仅 9.3M 非零;校验显示生成帧与传统模拟器一致,比真机慢 150-290 倍。","把一台红白机塞进大模型的权重里,是什么体验?开发者 Drew Breunig 的新开源项目 [nesformer](https:\u002F\u002Fgithub.com\u002Fdbreunig\u002Fnesformer) 给出了答案:一个标准的 Qwen3.5 checkpoint,用 `Qwen3_5ForCausalLM.from_pretrained` 就能正常加载,不需要任何自定义代码——而仓库首页的 GIF 里,这些权重跑起来的正是《超级马里奥兄弟》1-1 关。红白机的 6502 CPU、2KB 工作内存、32KB 卡带程序 ROM,全部住在模型权重和它的内部状态里。\n\n最反直觉的一点:这些权重没有一个经过训练。每一颗都是编译器算出来的——不是梯度下降的产物,是「编译」的产物。玩游戏时,模型逐 token 预测 CPU 的执行日志,大多数指令一个 token 搞定;宿主程序负责机器以外的一切:渲染画面、读手柄、触发 vblank 中断。\n\n## DeltaNet 的状态矩阵就是内存\n\n这个项目站在 Qwenputer 的肩膀上:Divyansh Agrawal 先把 CHIP-8、再把 RISC-V CPU 编译进 Qwen3.5-4B,nesformer 复用了它的编译器,并把核心戏法推到红白机规模——**模型的 Gated DeltaNet 状态就是机器的内存**。每个 DeltaNet 头维护一个 128×128 的状态矩阵,delta 规则每个 token 更新一次;把门控常开、用 one-hot 键,一次更新就精确覆写矩阵的一行,one-hot 查询再把它读回来。于是每一行都是一个存储单元,卡带、内存、寄存器全住在这些行里。\n\ncheckpoint 的规格单很有意思:沿用 Qwen3.5-4B 的头形状(16 个 key 头、32 个 value 头、维度 128)和 9216 的 MLP 宽度,7 层 Gated DeltaNet 加 1 层全零注意力层,隐藏维 768,词表 263,808。总参数 466.6M,bf16 下占 933MB,但非零的只有 9.3M——压缩后整个权重包只有 3.3MB。七层各有分工:第一层的状态存 PC 锚点、A\u002FX\u002FY 寄存器和标志位,第三层取指令,第五层读写数据,第七层吐出下一个 token。真实游戏代码平均每条指令只要 1.1-1.3 个 token。\n\n## 慢 150-290 倍,但对得离谱\n\n速度当然是玩具级的:numba 引擎在单核 CPU 上约 1650 token\u002Fs,numpy 引擎约 210,stock transformers 只有约 10——而真机需要约 70 万 token\u002Fs。宿主还会识别空转的等待循环,把 token 消耗砍掉 2.6-4.2 倍,但算下来整体仍比真机慢 150-290 倍:一帧要 1-6 秒,600 帧(10 秒的游戏时间)得跑约 25 分钟。\n\n正确性倒是拉满。README 报告的自由运行校验里,权重生成的 trace 与参考 CPU 完全一致:测试 ROM Thwaite 跑了 1,916,527 个 token、600 帧,保存下来的 300 帧与常规 6502 模拟器逐帧一致;另一个 ROM Full Game 跑了 2,645,885 个 token、420 帧,保存的 210 帧全部对上。限制也交代得诚实:只支持 NROM 卡带(mapper 0,32KB 程序 ROM),遇到非官方指令直接 HALT,图形由宿主逐扫描线渲染、不进权重,声音则完全没有。\n\n## 所以呢\n\n这个项目真正值得记的不是「AI 会玩马里奥」——它一次都没「学会」过游戏,这些权重只是被编译成了一台机器。值得记的是它用极端方式演示了一件事:Gated DeltaNet 这类线性注意力的循环状态是一块真正可寻址的内存,delta 规则就是写指令。权重不必只能从数据里学出来,编译器也能「写」模型。对研究模型内部能承载多少结构化状态的人,这是一个漂亮的上限实验;对看热闹的人,项目 MIT 协议、目前 229 星,clone 下来就能在自己电脑上跑。\n\n当权重可以被「编译」出来,训练就不再是造出模型的唯一路径——这大概是 nesformer 留下的又酷又扎心的问题。前身项目 [Qwenputer](https:\u002F\u002Fgithub.com\u002Fdivagr18\u002Fqwenputer) 同样开源,值得一并收藏。","https:\u002F\u002Fgithub.com\u002Fdbreunig\u002Fnesformer","998df6db-96e6-4b8e-8be1-cfa00a6cd177",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"41ceb665-4c22-4164-8f1a-efa7be1c94de","en","NES CPU inside Qwen3.5 weights: compiled, never trained","nesformer compiles a NES 6502 CPU, 2 KB RAM and cartridge ROM into a stock Qwen3.5 checkpoint. No training — a compiler wrote every weight.","What does it feel like to stuff an entire NES console into the weights of an LLM? Developer Drew Breunig's new open-source project [nesformer](https:\u002F\u002Fgithub.com\u002Fdbreunig\u002Fnesformer) answers that: a standard Qwen3.5 checkpoint that loads with `Qwen3_5ForCausalLM.from_pretrained` and needs no custom code — and in the GIF on the repo's front page, those weights are running Super Mario Bros. World 1-1. The NES's 6502 CPU, its 2 KB of work RAM, and 32 KB of cartridge program ROM all live inside the model's weights and internal state.\n\nThe most counterintuitive part: none of these weights were ever trained. A compiler computed every one of them. When a game runs, the model predicts the CPU's execution log one token at a time — most instructions take a single token — while a small host program plays the rest of the console around it: drawing the picture, reading the controller, firing the vblank interrupt.\n\n## The DeltaNet state matrix is the memory\n\nThe project stands on the shoulders of Qwenputer, in which Divyansh Agrawal first compiled a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B. nesformer reuses that compiler and pushes the central trick to console scale: **the model's Gated DeltaNet state is the machine's memory**. Each DeltaNet head keeps a 128×128 state matrix that the delta rule updates at every token. With the gate held open and a one-hot key, an update overwrites exactly one row of the matrix, and a one-hot query reads it back — so every row is a memory cell, and the cartridge, RAM and registers all live in rows.\n\nThe checkpoint's spec sheet is fun reading: it reuses Qwen3.5-4B's head shapes (16 key heads, 32 value heads, dim 128) and MLP width of 9,216, with 7 Gated DeltaNet layers plus one all-zero attention layer, hidden size 768, and a 263,808-token vocabulary. Total parameters: 466.6M — 933 MB in bf16 — but only 9.3M are nonzero, so the whole weights package zips to 3.3 MB. The seven layers split the labor: layer 1's state holds the PC anchor, the A\u002FX\u002FY registers and the flags; layer 3 fetches instructions; layer 5 reads and writes data; layer 7 emits the next token. Real game code averages just 1.1-1.3 tokens per instruction.\n\n## 150-290x slower than the real thing — and exactly right\n\nSpeed is toy-grade, of course. The numba engine runs at about 1,650 tokens\u002Fs on one CPU core, the numpy engine about 210, and stock transformers about 10 — while a real NES would need roughly 700,000 tokens per second. The host also recognizes idle wait loops and cuts token consumption by 2.6-4.2x, but the whole thing still lands 150-290x slower than the real console: a frame takes 1-6 seconds, and 600 frames — ten seconds of NES time — take about 25 minutes.\n\nCorrectness, though, is maximal. In the free-running checks reported in the README, the traces generated by the weights are identical to a reference CPU: the test ROM Thwaite ran 1,916,527 tokens across 600 frames, and all 300 saved frames match a conventional 6502 emulator exactly; the ROM Full Game ran 2,645,885 tokens across 420 frames, with all 210 saved frames matching. The limitations are stated honestly, too: NROM cartridges only (mapper 0, 32 KB of program ROM), unofficial opcodes emit HALT, graphics are rendered scanline-by-scanline by the host rather than living in the weights, and there is no sound at all.\n\n## So what\n\nThe real takeaway is not \"AI can play Mario\" — the model never \"learned\" the game; its weights were simply compiled into a machine. The takeaway is an extreme demonstration of something deeper: the recurrent state of linear-attention architectures like Gated DeltaNet is genuinely addressable memory, and the delta rule is a write instruction. Weights don't have to be learned from data — a compiler can \"write\" a model too. For researchers asking how much structured state a model can carry inside, this is a beautiful upper-bound experiment; for everyone else, the project is MIT-licensed, currently at 229 stars, and runs on your own machine after a clone.\n\nWhen weights can be compiled rather than trained, gradient descent stops being the only road to a model — which may be the most deliciously uncomfortable question nesformer leaves us with. Its predecessor [Qwenputer](https:\u002F\u002Fgithub.com\u002Fdivagr18\u002Fqwenputer) is open-source too, and worth a bookmark.","nesformer-qwen-nes-cpu-weights","2026-10-31T17:30:00Z","2026-10-31T17:15:27.939847Z","2026-10-31T17:15:27.939856Z",true,"agent",75,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"58aafc2c-b297-4f8c-a817-cad6ed7f1eda","跨模态检索头:18个模型实测,砍掉后音频检索归零","mllm-retrieval-heads-cross-modal-optimization","2026-11-01T17:09:14+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"8a550513-ef18-4d19-b609-d5fee5a853af","LeCun证明柏拉图表示定理:109M JEPA对齐胜72B","lecun-platonic-representation-theorem-jepa","2026-10-30T23:11:14+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"a491d4db-2d5a-4723-b093-28d1512fd647","VLA-Δ:物理状态变化当监督信号,LIBERO 98.4","vla-delta-physical-state-transition","2026-10-20T19:07:27+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"fd29c2f1-175f-416e-bde2-95dd65ca3465","KronLSTM 精准改写记忆:线性注意力长上下文新解","kronlstm-kronecker-delta-rule","2026-10-20T15:06:16+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"50687317-89b7-4857-ad66-86e2e251ff9d","阿里开源 OpenGCUA：8 个前沿模型电脑操控平均涨 19.6 分","opengcua-open-computer-use-stack","2026-10-18T13:10:17+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"36e4f28f-9176-485c-92b2-35cd0aac04fc","Apple MLX 联合 Meta 把 816B 模型装进 Mac Studio 集群","apple-mlx-muse-spark-mac-studio-thunderbolt-rdma","2026-10-16T03:00:00+00:00"]