[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-gpt6-astra-weight-space-programming":3,"topics-all":38,"news-related-fe609cac-33ac-4ea5-a6b7-db1a5913ec71":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"fe609cac-33ac-4ea5-a6b7-db1a5913ec71","GPT-6 徒手改权重:Qwen 数学7.9到48.1","MIT CSAIL 团队提出权重空间编程:让前沿模型直接读写神经网络权重。GPT-6 Astra 看原始权重就能认出网络在玩哪款游戏,不靠数据与梯度写出远胜 PPO 的策略,还为 Qwen3.5-0.8B 定位思考循环病灶,只改 0.016% 权重就让数学成绩从 7.9 分升到 48.1 分,全程无训练循环。","不给梯度、不跑训练循环,只让 GPT-6 Astra 把一个语言模型的权重当文本读,再动笔改数——MIT CSAIL 的 Rikiya Takehi、Jayden Teoh、Alex Zhang、Omar Khattab 四人 11 月 2 日发布博客,把这套玩法命名为 Weight-Space Programming(权重空间编程)。最硬的一组数字是:Qwen3.5-0.8B 的 MATH-500 从 7.9% 升到 48.1%,而实际改动的权重只有 131,072 个,占全模型 0.8B 参数的 0.016%。\n\n## 这不是微调,是读病历、动手术\n\n作者给 Astra 的工具是一个装着模型权重的 Python 会话:可以读权重、跑诊断前向,但拿不到任何梯度信息。练习材料只有 GSM8K 的 7,469 道题和 MATH 的 7,275 道(都不是测试集),预算是 10、25 或 50 次前向。Astra 自己定位到 Qwen3.5-0.8B 官方模型卡里警告过的「思考循环」问题:循环方向集中在第 19 层 MLP,最强的写入者是这一层的 11 号神经元。把相关方向移除之后,MATH-500 从 7.9% 拉到 48.1%,衡量指令遵循的 IFEval 从 21.0% 升到 62.3%——数学之外的收益属于顺带溢出。全部改动量:131,072 个权重。\n\n## 读权重:从 17,253 个数里认出一款游戏\n\n读侧实验从玩具网络起步。四个网格版游戏(Breakout、Pong、吃豆人、Freeway)用完全相同的 MLP 结构(500→32→32→5,共 17,253 个权重)、各以 PPO 训练 10 亿步。只把原始权重作为文本喂进去、不给候选选项,Astra 四个全中;对 Breakout 它给出 99% 置信度,推理过程里明确指出第 8 行的权重画出了底部挡板、第 1 至 3 行是砖块阵。扩散模型同样奏效:三个 DDPM 分别学画圆、方、三角,Astra 从输出层权重的几何结构里读出形状,还把实心图形画成了轮廓——作者认为这个「错误」恰好说明它读的是权重结构,而不是偷偷算了前向。更苛刻的一局:只给 7.2M 字符级 transformer 的中间四层,掐掉输入输出层,模型根本没法在文本上运行。Astra 不仅认出训练语言(Python、罗马字日文、英文),还把全部「复制头」逐个点名,与实测位置完全一致,零误报。\n\n## 写权重:零训练,一杆 517 分\n\n写侧更激进。Kinetix 环境里,一辆 PPO 训了 10 亿步的车卡在坡前出不来(局部最优)。Astra 一次作答、不许用工具,改掉 171,664 个权重里的 6,073 个(3.5%),随后 1,024 个测试回合全部通关。从零写更夸张:MinAtar 三款游戏,只给规则源码和一个全零初始化的网络,没有数据、损失、梯度、工具,连环境试跑都不允许,Astra 直接把整张权重表当数写出来,写完直接上考场。Breakout 拿到 517.4 分,PPO 训 1 亿步只有 15.8 分,训 5 亿步也只有 25.3 分——对后者是 20 倍差距;Seaquest 达到 5 亿步 PPO 的六成,Asterix 接近持平。\n\n## 作者自己划的边界\n\n这篇博客诚实的地方在于把失败案例写足了。Gemma 3 270M 两个副本,一个被微调成「谄媚」(用户说什么就附和什么),只看权重 Astra 分不出来,要加上注意力模式工具才能三对全部命中。更关键的是反向结果:Gemma 3 270M 本身很少循环(8.5% 的回答撞 token 上限,Qwen3.5 是 93%),Astra 对它无从下手,MATH-500 从 6.8% 到最高 7.7%,作者直言这点提升不成立。他们的结论写得清楚:权重编辑能搬开遮住已有能力的障碍,但补不出模型本来没有的能力。\n\n## 所以呢\n\n作者的前瞻判断是:深度学习此前一直把归纳偏置压在 Adam、Muon 这类微型符号优化器里,接下来,权重会被会推理的 LLM 原生优化。这既是优化问题也是可解释性问题——Astra 读权重的过程,本身就是一次白盒归因。工件已经放出:改过的模型在 Hugging Face(RikiyaT\u002FQwen3.5-0.8B-weight-space-programming),实验日志在 GitHub(RikiyaT\u002Flm_understand-logs),另有在线 demo 可玩。当优化器本身是个会推理的模型,「训练」和「编程」的边界该画在哪?原文:[Sparks of Weight-Space Programming](https:\u002F\u002Frikiyat.github.io\u002Fblog\u002Fweight-space-programming\u002F)","https:\u002F\u002Frikiyat.github.io\u002Fblog\u002Fweight-space-programming\u002F","59e2a320-9f20-4321-ad13-0d167de1b012",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"baf131c1-687a-49f4-87f6-4dd87c1c692f","gpt",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"bf49d9d5-9cdf-48a1-aa36-86d0e43675b1","en","GPT-6 Edits Weights by Hand: Qwen Math 7.9 to 48.1","MIT CSAIL: GPT-6 Astra reads and writes network weights - IDs games from raw weights, beats PPO without training, lifts Qwen3.5 math 7.9% to 48.1%.","No gradients, no training loops — just hand GPT-6 Astra a language model's weights as text and let it read and rewrite them. That is what Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab of MIT CSAIL call Weight-Space Programming, in a blog post published November 2. The headline result: Qwen3.5-0.8B's MATH-500 score climbs from 7.9% to 48.1%, with only 131,072 weights touched — 0.016% of the model's 0.8 billion parameters.\n\n## Not fine-tuning — diagnosis, then surgery\n\nAstra's tool is a Python session with the weights loaded: it may read them and run diagnostic forward passes, but it gets no gradient information. Practice material is 7,469 GSM8K problems and 7,275 MATH problems (never the test sets), with a budget of 10, 25, or 50 forward passes. Astra homes in on the thinking-loop problem that Qwen's own model card warns about: the loop direction concentrates in layer 19's MLP, with neuron 11 of that layer as its strongest writer. Removing the direction lifts MATH-500 from 7.9% to 48.1%, and IFEval, which measures instruction following, rises from 21.0% to 62.3% — a spillover gain beyond math. Total edit size: 131,072 weights.\n\n## Reading weights: naming the game from 17,253 numbers\n\nThe reading experiments start with toy networks. Four grid games (Breakout, Pong, Pac-Man, Freeway) share the exact same MLP shape (500→32→32→5, 17,253 weights), each trained with PPO for 1 billion steps. Fed only the raw weights as text, with no list of candidate games, Astra names all four correctly; for Breakout it reports 99% confidence and its reasoning explicitly traces the bottom paddle to row 8 weights and the brick rows to rows 1–3. Diffusion models work too: three DDPMs trained to draw circles, squares, and triangles are read from the geometry of the output layer, and Astra draws each shape as an outline — a mistake the authors take as evidence that it reads weight structure rather than secretly computing forward passes. The harshest round: only the middle four transformer blocks of a 7.2M character-level model are shown, with input and output layers stripped away, so the network cannot run on any text. Astra still names the training language (Python, romaji Japanese, English) and, in both attempts, identifies every copying head — matching the measured positions exactly, with zero false positives.\n\n## Writing weights: zero training, 517 points in one shot\n\nThe writing side is more radical. In a Kinetix environment, a car trained with PPO for 1 billion steps is stuck at a ramp (a local optimum). In a single reply with no tools, Astra changes 6,073 of the network's 171,664 weights (3.5%), and the car then clears all 1,024 test episodes. Writing from scratch goes further: across three MinAtar games, given only the rules, the source code, and a network initialized to all zeros — no data, no loss, no gradients, no tools, not even a trial rollout — Astra writes the full weight tables as text, and the network goes straight to evaluation. Breakout scores 517.4, versus 15.8 for PPO trained 100 million steps and 25.3 at 500 million steps — a 20x gap against the latter; Seaquest reaches about 60% of 500M-step PPO, and Asterix comes close to parity.\n\n## The limits, as the authors state them\n\nThe post is honest about failures. Given two fine-tuned copies of Gemma 3 270M — one trained to be sycophantic, echoing whatever the user suggests — Astra cannot tell them apart from weights alone; only with an attention-pattern tool does it pick the sycophant in all three pairs. More telling is the reverse result: Gemma 3 270M rarely loops (8.5% of answers hit the token limit, versus 93% for Qwen3.5), and Astra cannot improve it — MATH-500 moves from 6.8% to at most 7.7%, a gain the authors themselves call not established. Their conclusion is blunt: weight editing can remove an obstacle that hid an ability the model already had; it cannot add an ability the model lacks.\n\n## So what?\n\nThe authors' forward-looking claim: deep learning has squeezed its inductive biases into tiny symbolic optimizers like Adam and Muon, and soon our weights will be natively optimized by LLMs that reason about them directly. This is an optimization story and an interpretability story at once — Astra reading weights is itself a form of white-box attribution. Artifacts are out: the edited model is on Hugging Face (RikiyaT\u002FQwen3.5-0.8B-weight-space-programming), experiment logs are on GitHub (RikiyaT\u002Flm_understand-logs), and an online demo is linked from the post. When the optimizer is itself a reasoning model, where exactly does the boundary between training and programming sit? Source: [Sparks of Weight-Space Programming](https:\u002F\u002Frikiyat.github.io\u002Fblog\u002Fweight-space-programming\u002F)","gpt6-astra-weight-space-programming","2026-11-02T23:09:36Z","2026-11-02T23:09:39.494651Z","2026-11-02T23:09:39.494663Z",true,"agent",173,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"6444b62a-663a-4bad-9b7a-6e3a901292d7","读权重不稀奇了:笔记本27B猜中3局,论文魔术遭拆解","qwen-27b-local-weight-reading","2026-11-04T17:11:51+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"13d6adf3-a5c3-46fb-9ff5-a6999dcf0300","Critic值轮内近乎恒定:MPO重构Agent信用分配","mpo-turn-level-credit-long-horizon-agents","2026-10-28T19:08:42+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"48726cf0-6f0c-4b09-841e-06aa8b0ffcfd","上海AI Lab Intern-Decision:多模态决策模型,4B 微超 Jev 90.02%","intern-decision-qwen3-5-multimodal-decision-model","2026-10-18T11:13:06+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"9fcb5d74-315e-4d57-a7da-290fbfb9e91c","阿里 Qwen 发布弹性 RL 框架 QwenGyre,训练长时 Agent 提速 1.85x","alibaba-qwen-qwengyre-elastic-rl-xlong-agents","2026-10-14T00:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"206ea36a-1eea-463f-a241-1e3b32f5ec2d","USTC GraphForge:证据图把任务和 rubric 钉在一起,Qwen3.6-27B 涨三基准","graphforge-ustc-qwen36-27b-evidence-graph","2026-10-04T03:05:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"21c7dec1-f68e-4641-974c-ae2bce87393e","教师打分、验证器掌舵:腾讯混元 FlowBalance 给自蒸馏装上方向门控,Qwen3-8B 数学均值超 GRPO 2.12 分","flowbalance-verifier-gated-self-distillation","2026-09-08T15:08:17+00:00"]