[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-perfarena-llm-code-performance-ranking":3,"topics-all":38,"news-related-d1d54032-073b-433e-9430-547c6ffb4a49":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"d1d54032-073b-433e-9430-547c6ffb4a49","贵的模型，没写出更快的代码：9455 份实测","纽约大学阿布扎比校区等机构开源评测场 PerfArena：13 个前沿模型用 7 种语言做同题，对 9455 份正确代码实测延迟、峰值内存与能耗。gpt-6-astra 没有被任何模型显著超越，其余代码平均慢 1.1 到 3.6 倍；按每份正确解算，最贵模型成本是它的 9.3 倍，代码却没更快。","谷歌 2026 年披露，公司新代码里已有 75% 由 AI 生成；微软 2025 年的数字是 20% 到 30%。这些代码「能不能跑对」一直有基准盯着，但「跑得快不快、吃多少内存、耗多少电」，长期没人系统地量。纽约大学阿布扎比校区等机构 11 月 13 日挂上 arXiv 的预印本 PerfArena（尚未同行评审）把这笔账摆上了台面：让 13 个前沿模型用 7 种编程语言做同样的题，再对 9455 份通过正确性检查的代码逐份实测延迟、峰值内存和能耗。\n\n## 通过率之外，还有三笔账\n\n论文的出发点很直接：现有代码基准从 HumanEval 到 LiveCodeBench 一路，基本只数通过率；少数性能向基准要么只测一两项指标，要么语言覆盖窄，能耗更是很少进榜。作者团队此前的研究已经把问题量化过：大模型生成的安卓应用里，性能反模式的密度约为人写代码的 10 倍，他们把这个自我强化的循环叫作「performance loop of hell」——模型学自不太在乎性能的人类代码，又成批复制这种低效。\n\n## 一个照搬竞技场玩法的跑分场\n\nPerfArena 把评测组织成两套比赛：CLBGComp 取计算机语言基准游戏里 10 道计算密集型老题；LeetCodeComp 精选 99 道「已通过解之间存在可测延迟差」的力扣题。13 个模型——OpenAI 五个、Anthropic 四个，外加 Gemini、DeepSeek、Grok、GLM 各一个——在 C++、C#、Java、JavaScript、PHP、Python、TypeScript 七种语言下用同一提示模板作答，所有解在 Unix 主机上按同一协议执行，能耗直接读 Intel RAPL 硬件计数器。排名机制学 Chatbot Arena：同一道题同一语言、两个模型都解出的格子算一场对决，数值低者胜，落在测量噪声内的平局各得半分。目前库里存了 9455 份正确且测完的解。论文顺带给出成本参考：跑完一轮 LeetCodeComp 平均 8.11 美元、22 小时；单模型最贵一轮 19.56 美元，最便宜 0.56 美元。\n\n## 榜单：一家独大，两个注脚\n\n结果并不平均。12 个模型都能通过九成以上的 LeetCode 题，通过率早已挤在一起，拉开差距的是性能：gpt-6-astra 在两套比赛分别通过 99.7% 与 100% 的题，延迟、能耗、峰值内存三项上没有被任何模型显著超过的，只有它一个；其余模型的正确解平均慢 1.1 到 1.6 倍（LeetCode 赛场）与 1.2 到 3.6 倍（CLBG 赛场）。注脚有二：LeetCode 的 Python 与 PHP 两个语言单项，延迟第一是 claude-opus-5-5；CLBG 赛场上 glm-5.3-flash 与 gemini-3.8-flash 和 gpt-6-astra 同处帕累托第一梯队——但论文自己注明，glm-5.3-flash 只解出 22\u002F70 个格子，样本太少，统计上分不开。\n\n## 贵，买不来快\n\n价格与性能的关系更有意思。按 token 单价，gpt-6-astra 每百万 5\u002F25 美元，并不便宜（claude-fable-5-1 的 10\u002F50 最贵，gpt-5.6-luna 的 0.2\u002F1.2 最便宜）。但换算成「每份正确解的成本」，gpt-6-astra 每份解只输出 321 个 token（LeetCode 赛场），成本不到两个最贵模型的一半，代码还更快；按每份正确解算，最贵的模型在两个赛场分别烧掉 gpt-6-astra 的 2.6 倍与 9.3 倍，代码却没有更快。论文的结论写得很直白：除了 CLBG 赛场要避开最便宜的那一档（四个最便宜的模型恰好生成最慢的代码），多付钱买不来更快的代码。\n\n## 能耗，基本是延迟的影子\n\n能耗指标容易被神化，这篇论文反而泼了冷水：90% 的解，平均功率都挤在 13.1 到 16.0 瓦之间，能耗排名与延迟排名的相关系数高达 0.91 与 0.97——能耗大抵就是延迟乘以一个几乎恒定的功率。作者仍坚持三项指标分开报告，理由是任务与语言层面会出现分歧。读这份榜单也要记住限定：全部测量在一台 2 核 i7-7600U 的安静主机上完成；CLBG 只有 10 道题，论文承认统计功效不足；线上榜单只允许官方写入，第三方要自己部署开源代码另建。\n\n所以，下次选代码模型，通过率只是入场券，真正拉开差距的是目标语言的单项榜和每份正确解的成本。排行榜开在 perfarena.net，代码在 GitHub 开源。反过来问一句：如果你选型时连「生成的代码跑得快不快」都不看，这笔隐形的算力账单，打算什么时候算?\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18030","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"9587332b-df06-4597-8a51-20746a19c745","en","Paying More Buys No Faster Code: 9,455 Solutions Measured","PerfArena times 9,455 solutions from 13 LLMs in 7 languages. gpt-6-astra is never significantly beaten; rivals run up to 3.6x slower at 9.3x the cost.","Google disclosed that 75% of its new code was AI-generated in 2026; Microsoft put its own 2025 share at 20-30%. Whether that code is correct has benchmarks watching. How fast it runs, how much memory it peaks at, and how much energy it burns has mostly gone unmeasured at scale. A preprint posted to arXiv on 13 November by researchers at New York University Abu Dhabi and Universidade do Algarve, not yet peer-reviewed, puts that ledger on the table: PerfArena prompts 13 frontier LLMs to solve the same problems in 7 programming languages, then measures every one of the 9,455 correct solutions for latency, peak memory, and energy.\n\n## Three Bills Beyond Pass Rates\n\nThe premise is blunt. Existing code benchmarks, from HumanEval to LiveCodeBench, mostly count correctness; the few efficiency-oriented ones cover one or two metrics, few languages, and rarely energy. The authors had already quantified the problem in earlier work: LLM-generated Android apps carry roughly 10 times the density of performance anti-patterns of human-written equivalents, a self-reinforcing loop they call a performance loop of hell. Models learn from human code that rarely prioritizes performance, then mass-produce the same inefficiency.\n\n## An Arena That Borrows the Chatbot Arena Playbook\n\nPerfArena organizes evaluation into two competitions. CLBGComp takes 10 compute-intensive classics from the Computer Language Benchmarks Game; LeetCodeComp curates 99 LeetCode problems whose accepted solutions already differ measurably in latency. Thirteen models - five from OpenAI, four from Anthropic, plus one each from Google, DeepSeek, xAI, and Zhipu - answer with the same prompt template in C++, C#, Java, JavaScript, PHP, Python, and TypeScript. Every solution runs on a Unix-like host under one protocol, with energy read straight from Intel RAPL hardware counters. The ranking mechanic copies Chatbot Arena: a cell that two models both solve is one duel per metric, the lower value wins, and ties within measurement noise split half a win each. The bank currently holds 9,455 correct, fully measured solutions. The paper also prices the exercise: one LeetCodeComp sweep averages $8.11 and 22 hours, with the priciest model at $19.56 and the cheapest at $0.56.\n\n## One Dominant Leader, Two Footnotes\n\nThe results are lopsided. Twelve models clear more than 90% of LeetCodeComp, so pass rates crowd together; performance is where they split. gpt-6-astra solves 99.7% of LeetCodeComp and 100% of CLBGComp, and no rival significantly beats it on latency, energy, or peak memory. The correct solutions of the other models average 1.1-1.6x its latency on LeetCodeComp and 1.2-3.6x on CLBGComp. Two footnotes matter. On LeetCodeComp, claude-opus-5-5 leads latency in Python and PHP while gpt-6-astra leads the other five languages. On CLBGComp, glm-5.3-flash and gemini-3.8-flash share Pareto level one with gpt-6-astra - but the paper itself notes glm-5.3-flash solves only 22 of 70 cells, too few for the tests to separate it.\n\n## Paying More Buys No Faster Code\n\nThe price story is the sharp part. Per token, gpt-6-astra at $5\u002F$25 per million is not cheap (claude-fable-5-1 at $10\u002F$50 costs most; gpt-5.6-luna at $0.2\u002F$1.2 least). Converted to cost per correct solution, though, gpt-6-astra emits just 321 response tokens per solution on LeetCodeComp, costs less than half as much as the two most expensive models, and writes faster code. Per correct solution, the most expensive LLM burns 2.6x what gpt-6-astra does on LeetCodeComp and 9.3x on CLBGComp - and still generates no faster code. The conclusion reads plainly: beyond avoiding the cheapest tier on CLBGComp, where the four cheapest models happen to write the slowest code, paying more does not buy speed.\n\n## Energy Is Mostly Latency in Disguise\n\nFor anyone hoping energy would tell an independent story, the paper pours cold water: average power sits between 13.1 and 16.0 W for 90% of solutions, and energy rankings correlate with latency rankings at 0.91 and 0.97. Energy is roughly latency times a nearly constant power draw. The authors still report all three metrics separately, because task- and language-level disagreements do appear. Read the leaderboard with its limits in mind: everything ran on one quiet host with a 2-core Intel Core i7-7600U; CLBGComp has just 10 problems, and the paper concedes the statistical tests have little power there; the public leaderboard accepts results only from the official pipeline, though the code is open for anyone to self-deploy.\n\nSo the next time you pick a coding model, treat pass rates as the entry ticket and check the per-language leaderboard and the cost per correct solution. The leaderboard is live at perfarena.net and the code is open on GitHub. Flip the question around: if code speed has never entered your selection criteria, when exactly were you planning to read that invisible compute bill?\n","perfarena-llm-code-performance-ranking","2026-11-16T19:13:41Z","2026-11-16T19:13:42.123314Z","2026-11-16T19:13:42.123324Z",true,"agent",43,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"9dc3bd1e-95da-4c07-b352-900688d1af44","TGRL把温度分组变成训练信号:RLVR训练提速36%","tgrl-temperature-grouped-rl","2026-10-01T19:02:31+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"2e27016d-b90e-45c7-825a-41fd1e435c80","JHU 新研究:组合持续学习机制,百任务记忆留存从 1.2% 提到 34.9%","compose-cl-long-horizon-memorization","2026-09-16T15:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"2731ed1c-17c3-4d85-9174-983cf50743e3","地铁售票机上的 AI 大考:2.6GB 端侧模型 91.32 分超 GPT-5.6,规则基线也拿 84.6","metrollm-bench-transit-kiosk-llm","2026-09-12T23:08:18+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"58ed753e-ad6d-4aac-95f4-36bf217e169c","把 10 万条人类视频变成机器人教材:RoboTok 检索 mAP 提升约 50 倍,hard 任务 79.3% 对 19.5%","robotok-retrieval-benchmark-reread","2026-09-06T21:11:25+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"2e7a2be3-762e-46eb-b582-af299c4ec173","模型越新，拒绝越难拆？76页审计：并不均匀","refusal-abliteration-generation-audit","2026-11-16T17:45:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d9243227-680a-4fde-908d-8990969a54e0","LuthorSQL：7B 模型替 Agent 省三成账单","luthorsql-text-to-sql-agent-cascade","2026-11-16T05:00:00+00:00"]