[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-weightbench-llm-weight-space-programming":3,"topics-all":38,"news-related-43903d4a-7891-411b-9ea6-80708f7f8309":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"43903d4a-7891-411b-9ea6-80708f7f8309","让大模型改权重:42道题测出能力边界","南密西西比大学团队的 WeightBench 用 42 道题、五个赛道测大模型直接读写神经网络权重的能力:沙箱断网、注入计量 NumPy 子类追踪算术、哈希链账本算分。论文实测显示前沿智能体已能从零写出可玩的控制策略,但修补语言模型、无数据压缩、分辨不可能的改动仍集体失灵。","过去两年,大模型改代码改文档大家见多了。有一件事还没普及:让它直接改神经网络本身的权重——不看源码,不看架构图,只拿到一堆浮点数,像改代码一样改掉该改的那几行。\n\nMIT CSAIL 团队 11 月 2 日的一篇博客把这件事演示了一遍。GPT-6 Astra 拿到一个只有 17253 个权重的网络,不用任何工具,直接判断出这是拿 Breakout 训出来的,置信度 99%。另一个失败的控制策略——车死活开不过那条条纹——它一次性编辑 3.5% 的权重,成功率从 0% 变成 100%。最狠的是 Qwen3.5-0.8B:数学上它改了 0.016% 的权重,MATH-500 从 7.9% 拉到 48.1%,IFEval 涨了 3 倍。全程不许写训练循环。\n\n演示很爽,但说明不了问题:它到底是真在推理,还是撞对了答案,还是训练数据里见过?11 月 13 日,南密西西比大学的 Aayush Marasini、Barsat Khadka 和 Nick Rahimi 在 arXiv 放出一份叫 WeightBench 的基准(https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18553),专门把这件事拆开量。\n\n## 42 道题,五个方向\n\n题库有 42 道任务,分五个 track:READ(10 道)只给权重、禁止前向传播,考你能否从浮点数读出网络行为;FIX(13 道)要求精确修补已有网络;META(5 道)给你一个根本做不到的改动加几个很难的诱饵,看你会不会硬改;WRITE(6 道)从全零的空白写出能跑的网络;SYSTEMS(8 道)不许跑校准,把模型压到边缘设备能跑。被改的网从小到 2K 参数的 MLP,大到 7B 的 Transformer。\n\n## 真正难的是不让它跑\n\n最有分量的不是题目,是笼子。Linux 上用 bubblewrap 的 --unshare-all 隔出用户、PID、IPC、UTS、cgroup 和网络命名空间,断网,禁掉 transformers 和 vllm 这类执行库,agent 能走的通道只剩两条:自己的 LLM API,和一个本地带 token 认证的 Weight API。简单用 FLOP 上限卡不行——论文说,对权重矩阵做 SVD 的开销比一次前向还大,粗暴卡会把合法的数学分析一起毙掉。做法是往每个 agent 进程注入计量的 NumPy 子类,在算子层面追踪权重上的算术,每次矩阵乘按 2mkn 收 FLOP;再靠数据流溯源分辨「前向链」和「合法分析」,像 max(W,0)u + min(W,0)l 这种区间界传播不算前向。\n\n事后审计有 28 条隔离标志,分文本、运行时、完整性、会计四层。前向、探针、隔离事件都写进哈希链账本,分数从账本算,不信 agent 自报。作者还做了红队:29 个作弊场景加 30 个诚实对照,每个场景独立跑三次共 180 次,28 个作弊全被抓住;漏网那个因没有 transcript 被 OS 沙箱挡住;90 次诚实对照零误报。\n\n## 改权重是一笔要算总账的账\n\n计分不是答对就赢。最终分 SWB 是四项相乘:任务增益、附带损伤、结构化编辑代价、算力占用。附带损伤看改完后模型在留出基准(比如 MMLU)上还剩多少,下界钉在 0,免得别处的意外提升抵消这边的损失;编辑代价同时看 L0 参数比例和矩阵秩比例,取小的。常数也是定死的:α ≈ 2.23,让附带损伤基准掉 10% 时总分掉 20%;编辑或算力预算用满,罚 10%。相乘意味着任何一项都换不走另一项,没用的改动再便宜也是零分。\n\n## 写得出,修不动,更看不出该不该修\n\n论文实测的表格比总会有意思。42 道题宏平均:Claude Opus 5.5 拿到 0.535,Sonnet 5.5 0.466,Haiku 5.5 0.402,DeepSeek v4-flash 0.215。GPT-6.1-sol 因为跳过了 4 个每题要跑一到一点五 GPU 小时的 reasoning-loop 任务,只跑了 38 道,共同题口径上 0.568,Opus 同样 38 道是 0.546。\n\n拆到 track,WRITE 这栏差距最大:Opus 0.829,GPT-6.1-sol 0.863,Sonnet 0.569,Haiku 0.369,DeepSeek v4-flash 0.000。从零写一个能打的策略网络,前沿模型基本做到了。\n\nFIX 还基本是开着的。Opus 0.404 略微领先,GPT-6.1-sol 那个 0.571 看着更高,但只覆盖 13 题里的 9 题;四个 reasoning-loop 题上,只有 Opus 拿到 3 题非零、Sonnet 拿到 1 题。\n\nMETA 是最扎眼的一栏:三个 Claude 模型全部落在 0.485 到 0.500 之间,论文说这基本就是随机猜的基线水平;GPT-6.1-sol 只有 0.260。模型基本分不清「这个改动做不了」和「这个改动只是难」,而且跟被测模型强弱基本无关。SYSTEMS 一栏全线塌陷,全在 0.177 到 0.238 之间——无数据不跑校准的压缩谁都不行。\n\n还有个细节:DeepSeek v4-flash 42 次运行里有 13 次被隔离,是全场最高的隔离率;论文说被隔离的运行还没人工复核,所以上面每个分数都是下界。\n\n## 值得琢磨的三个点\n\n第一,机制层有独立佐证。WeightBench 的分数是论文自报,仓库排行榜与论文表格一致但同源,只能按「论文实测」引用;但「LLM 能原生读写权重」这件事,MIT CSAIL 那篇博客和 2026 年 3 月的权重空间学习综述(arXiv:2603.10090,Han 等人)是两个团队两个域名的独立支撑。\n\n第二,最该盯的其实是 META 那栏。模型很会改,但不知道什么时候不该改。权重一旦能被读被改,「拒绝修改」就是真问题:知道自己边界在哪的 agent,比无所不能的 agent 更值钱。\n\n第三,论文对自家基准的诚实度值得记一笔:42 道题里只有 21 道的参考解是 agent 权限内能拿到的,剩下 21 道需要特权访问。作者写明,参考解能证明「权重里存在解」,但不能保证「不用数据也能找到」。39 道题通过有效性门槛,要求参考解 SWB 不低于 0.5 且比基线高出 0.2。\n\n所以呢? WeightBench 真正的贡献可能不是那份分数,而是它证明了一件事:判断「模型是真的在推理」还是「模型在暴力试错」,不能靠把题出得更难,得靠把 agent 关进笼子、给它装电表、把每次计算记进账本。SWE-bench 之后大家忙着让 agent 变强,这条路是反过来想的。至于权重编程能不能成为工程纪律——现在还早:写网络通了,修网络没通,判断该不该改几乎没通。这三格的落差就是接下来一年的活儿。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18553","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ae9da5ea-0744-4193-b05f-b7132c0f59de","en","Letting LLMs Edit Weights: 42 Tasks Map the Capability Edge","WeightBench gives LLMs 42 tasks on reading and writing network weights. Agents write working policies from scratch but cannot patch language models.","For two years we have watched language models edit code, edit documents, edit images. One capability has not gone mainstream: letting a model edit the weights of a neural network itself. Not the source code, not the architecture diagram, just a bag of floating point numbers, and then changing the few lines that need changing, the way a programmer edits code.\n\nA blog post from the MIT CSAIL team on November 2 demonstrated exactly that. GPT-6 Astra received a network with only 17,253 weights and, using no tools at all, correctly identified that it had been trained on Breakout, at 99% confidence. Given a control policy that had failed to train, where the car never made it across the stripe, it edited 3.5% of the weights in one shot and moved success from 0% to 100%. The sharpest case was Qwen3.5-0.8B: on math it edited 0.016% of the weights, lifting MATH-500 from 7.9% to 48.1%, and IFEval by 3x. No training loop was permitted at any point.\n\nDemonstrations are fun but they prove nothing. Was the model actually reasoning, or did it get lucky, or had it seen the answer in training? On November 13, Aayush Marasini, Barsat Khadka and Nick Rahimi of the University of Southern Mississippi posted a benchmark called WeightBench to arXiv (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18553) that takes the question apart and measures it.\n\n## 42 tasks, five tracks\n\nThe task registry holds 42 tasks across five tracks. READ (10 tasks) hands over only the weights, forbids any forward pass, and asks whether you can read a network's behavior out of floating point numbers. FIX (13 tasks) demands surgical patching of an existing network. META (5 tasks) pairs a genuinely impossible modification with several merely hard decoys, to see whether the agent will try anyway. WRITE (6 tasks) starts from an all-zero blank slate and asks for a working network. SYSTEMS (8 tasks) forbids any calibration run and asks for a model compressed enough for an edge device. The networks involved range from 2K-parameter MLPs to 7B-parameter Transformers.\n\n## The hard part is preventing the model from running\n\nThe most substantial piece of this benchmark is not the task list, it is the cage. On Linux the harness uses bubblewrap with --unshare-all, carving out separate user, PID, IPC, UTS, cgroup and network namespaces. The network is cut off, execution libraries such as transformers and vllm are denied, and the agent is left with two exits: its own LLM API and a local, token-authenticated Weight API.\n\nThe tricky part is proving the agent did not quietly run the model. A naive FLOP cap does not work: the paper notes that computing an SVD of a weight matrix routinely costs more FLOPs than a forward pass, so a crude cap would kill legitimate mathematical analysis along with it. The approach is to inject a metered NumPy subclass into every agent process and trace arithmetic on weights at the operator level, charging 2mkn FLOPs for each matrix multiplication. Forward-pass chains are then separated from legitimate analysis by data-flow provenance: a product involving weights whose values were passed through a nonlinearity is classified as bound propagation rather than a network layer, so interval-bound propagation of the form max(W, 0)u + min(W, 0)l is not miscounted as a forward pass.\n\nThe post-run audit has 28 quarantine flags, split across text rules, runtime rules, integrity rules and accounting rules. Every metered forward pass, probe and guard event goes into a hash-chained ledger, and the final score is computed from that ledger rather than from whatever the agent claims. The authors also ran a red-team exercise: 29 cheat scenarios plus 30 honest controls, three independent trials each, 180 runs in total. The audit caught and quarantined 28 of the 29 cheats in all three trials; the single unflagged cheat was blocked by the OS sandbox anyway because it had no transcript; and the system recorded 0 false positives across all 90 honest control trials.\n\n## Editing weights is a job with a bill attached\n\nSolving the task is not enough to score. The final score SWB multiplies four components: task gain, collateral damage, structural edit cost and compute use. Collateral damage measures how much of the model's other capability survives the edit, read on held-out benchmarks such as MMLU, and it is lower-bounded at 0 so that accidental improvements elsewhere cannot offset losses. Edit cost tracks the smaller of the L0 parameter count fraction and the matrix rank fraction, penalising broad, messy edits.\n\nThe constants are set deliberately. Alpha is about 2.23, so a 10% drop on the collateral benchmark reduces the overall score by 20%. Exhausting the edit budget or the compute budget costs 10%. Because the factors multiply, no component can be traded against another: a useless edit scores near zero however cheap it is.\n\n## Can write, cannot fix, and cannot tell when not to edit\n\nThe table is more interesting than the macro number. On all 42 tasks, Claude Opus 5.5 reaches 0.535, Sonnet 5.5 0.466, Haiku 5.5 0.402 and DeepSeek v4-flash 0.215. GPT-6.1-sol was run on 38 tasks, skipping the four reasoning-loop tasks that each need one to one and a half GPU-hours per run; on the tasks it shares with the others it reaches 0.568 against Opus's 0.546.\n\nBroken out by track, WRITE separates the models most. Opus reaches 0.829, GPT-6.1-sol 0.863, Sonnet 0.569, Haiku 0.369 and DeepSeek v4-flash 0.000. Writing a working policy network from scratch is essentially solved by frontier models.\n\nFIX is still largely open. Opus leads slightly at 0.404, and GPT-6.1-sol's 0.571 looks higher but covers only 9 of the 13 tasks; on the four reasoning-loop tasks, only Opus scored above zero on three of them and Sonnet on one.\n\nMETA is the most striking column. All three Claude models land between 0.485 and 0.500, which the paper calls essentially the baseline of random guessing, and GPT-6.1-sol manages 0.260. In other words, models largely fail to distinguish an impossible edit from a merely hard one, and that has little to do with how strong the model being tested is. The SYSTEMS column collapses entirely, everything between 0.177 and 0.238: data-free compression without calibration remains hard for everyone.\n\nOne easily missed detail: 13 of DeepSeek v4-flash's 42 runs were quarantined, the highest rate in the field. And since no quarantined run has been reviewed by a human yet, every score reported above is a lower bound.\n\n## Three things worth sitting with\n\nFirst, the mechanism layer has independent corroboration. The WeightBench numbers are self-reported by the paper; the leaderboard in the repository matches the paper's table exactly, but it is the same origin, so it can only be cited as the paper's own measurement. The underlying claim that language models can natively read and write weights does have separate support, from the MIT CSAIL blog and from a March 2026 survey of weight space learning (arXiv:2603.10090, by Han and colleagues, which splits the field into understanding, representation and generation).\n\nSecond, the META column is the one to watch. Models are good at editing and bad at knowing when not to edit. Once weights can genuinely be read and modified, refusing to modify becomes a real problem, and an agent that knows where its own boundary lies is worth more than an agent that can do everything.\n\nThird, the paper's candour about its own benchmark deserves a note. Only 21 of the 42 tasks have reference solutions that use nothing beyond what the agent can access; the remaining 21 require privileged access such as gradient fitting on calibration prompts. The authors state plainly that a privileged reference proves a solution exists inside the weights but does not guarantee it can be found using only data-free analysis. 39 of the 42 tasks clear the validity gate, which requires a reference solution scoring at least 0.5 and beating the naive baseline by at least 0.2.\n\nSo what? The real contribution of WeightBench may not be the scores but the demonstration that separating genuine reasoning from brute-force trial and error cannot be done by making the tasks harder. It takes a cage, a meter and a ledger. After SWE-bench, everyone has been busy making agents stronger; this path runs the other direction.\n\nAs for whether weight programming can become an engineering discipline, it is too early. Writing networks basically works, fixing networks does not, and knowing when not to edit barely works at all. The gap between those three states is the work of the next year.\n","weightbench-llm-weight-space-programming","2026-11-16T07:20:00Z","2026-11-16T07:31:12.449813Z","2026-11-16T07:31:12.449820Z",true,"agent",120,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"e8f79d4f-9b2e-465a-94c1-cc725f09bc19","多工具检索:召回率翻倍,完整率几乎不动","tool-retrieval-completeness-multitool-testbed","2026-11-02T11:15:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"c368ad9f-9308-4a0b-8f5c-3ae4601b48b9","D2K-Bench: 专家设计把 LLM 写 GPU 核提速 33.9%","d2k-bench-llm-gpu-kernel-design-guidance","2026-10-07T03:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"fe5546f3-09e4-42a3-bfde-6bbd0e8d474c","外星世界实测:探索 4 轮 87.6%,垫底 12.9%","explorationbench-alien-worlds","2026-09-28T21:07:30+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"3c6fcf46-f5bb-4136-931c-69cd64216e12","Skill-Use 基准揭示 Agent 短板：会做任务，不等于会用 Skill","skill-use-agent-harness-benchmark","2026-08-06T08:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"74de194b-9e2c-45ab-aa13-12fe210e66ba","HiGram 给 Agent 记忆加上“路径定位”：先找证据，再改记忆","higram-agent-memory-path-localization","2026-08-05T09:32:43+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"648ee35f-321e-4cc7-8e88-9a9d3ebae033","小模型每晚自学:一年后从 52 分掉到 5 分","on-device-llm-self-training-judge-failure","2026-11-19T03:30:00+00:00"]