[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-gvs5h-ledger-orchestration-qwen-fable-rerun":3,"topics-all":38,"news-related-278dc1cc-ab13-4de3-9ed4-27f6732dda3c":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"278dc1cc-ab13-4de3-9ed4-27f6732dda3c","五个27B组队靠账本编排追平Fable 5,复现只承认打平","账本式零样本自编排让五个27B实例靠共享文件分工,在代码难题基准上把通过率从66.8%推到92.4%,压过旗舰模型单次调用的90.4%。独立开发者换工具重跑前十题五轮均分9.8,与作者脚手架的9.6其实是同一个数字,运行记录全部公开。复现者自己承认样本只有十题。","一个开源方法在 Hacker News 上刷了屏。论文的评测说:把五个 Qwen3.8-27B 实例放进同一套共享文件账本里分工协作,就能在 LiveCodeBench 最难的 100 道题上把通过率从 66.8% 顶到 92.4%,超过闭源旗舰 Claude Fable 5 单次调用的 90.4%。一个月后,一位独立开发者用一套完全不同的工具重建了它,50 次运行里每一次的请求、回复、代码和花费全部公开,结论比原论文保守得多:打平,不是超越。[论文原文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.26480)\n\n## 那套方法到底做了什么\n\n论文的叫法是账本式零样本自编排,简称 GVS5H,作者解释成 Gorilla vs. 5 Humans。不训练任何编排器,每个角色都是同一个模型的一次全新调用,彼此不直接对话,只通过磁盘上的几个文件交换信息。题目写在 task.md,管理者把三到六句策略写进 plan.md,想法和中间结论写进 notes.md,当前解法是 solution.py。管理者每轮只派发一个任务,worker 干完重写这两个文件,再由一段跑公开样例的脚本判定成败。循环最多 10 轮,计划文件截到 4,000 字符,笔记截到 8,000 字符。\n\n评测用 LiveCodeBench release_v6 里按比赛日期最新的 100 道 hard 题,分数经修正后的评分器计算。论文自报的结果是:Qwen3.8-27B 单次 66.8%,加管理者后 92.4%,涨了 25.6 个百分点;Qwen3.8-Flash-Next 从 84.2% 到 93.0%,花的钱只要 Fable 5 的 9%;DeepSeek-V4.1-Flash 从 88.4% 到 92.0%;GPT-5.6-Terra 从 80.8% 到 88.0%,GPT-5.6-Luna 从 70.4% 到 81.2%。主条件是六个模型各跑五遍,其中五个有单次与编排两臂,Fable 5 只跑了单次;另有五个模型走 OpenRouter 网关,只跑一遍。\n\n## 复现者跑出了什么\n\n10 月 31 日,Memdoor 的作者用自己写的另一套工作流引擎重建了这套方法:同一个 Qwen3.8-27B、同一批提示词、同样 128K 上限、同样十轮,连发给模型的每条消息都逐字节对齐,他为此写了个对拍脚本。测的是作者的前 10 道题,跑五轮。\n\n结果:复现工作流均分 9.8,作者脚手架 9.6,Fable 5 单次 8.0。50 次运行解出 49 次,输出 15.3M token,共花 35.79 美元;加上被时钟掐断或被叫停的那 16 次,是 38.77 美元。单次运行中位 40 分钟模型时间,最长一次四小时,全部跑在 OpenRouter 上。[复现仓库](https:\u002F\u002Fgithub.com\u002Fguregodevo\u002Fmemdoor\u002Ftree\u002Fmain\u002Fexamples\u002Fworkflows\u002Fteam-solve)\n\n## 论文承认的三件事\n\n第一,收益不普适。论文写得很直白:管理者不是免费的,它的权衡可能落在比单次调用更差的答案上。Qwen3.6-35B 在关推理的条件下两头都是负的——它的构思阶段认出了和单次调用相同的那个正确优化,又自己说服自己那太容易出错,转去实现一个渐近更慢的方案,worker 还带了个 bug。那道 LCB 3765 题目的完整故事是:紧凑而正确的代码,被分解这件事搅乱了。\n\n第二,Fable 5 只跑了单次调用,因为太贵,它如果也配管理者能涨多少,论文说不知道。另外它 500 次里有 6 次直接拒答,论文按失败计分,没有偷偷换成替补模型。\n\n第三,账单要翻三倍。论文自己的成本表:Qwen3.8-27B 单次跑一题 18.95 美元、每解出一题 0.28 美元;加上管理者变成 47.82 美元和 0.52 美元。Fable 5 单次是 61.11 美元。结论里那句编排常常是模型规模的廉价替代品,是带着这个前提说的。\n\n## 复现者写的四条免责声明\n\n复现者比论文更谨慎,四条全写在仓库 README 里:测的是 10 题不是 100 题,样本这么小,9.8 和 9.6 其实是同一个数字,能说的只是和作者的脚手架打平;这些比赛是 2025 年 3 月 29 日到 4 月 6 日的,模型可能见过,污染排不掉;Fable 5 那一列来自作者仓库里的结果文件,没有在这个框架里跑过;循环只看得到公开样例,分数靠隐藏测试。\n\n最后一条,恰恰是 Hacker News 讨论里被追问最多的地方之一:拿编过排的 Qwen 去比没编排的 Fable 5。同一场讨论里还有人指出,LiveCodeBench 的题目横跨 2023 年 5 月到 2025 年 4 月,污染几乎难以避免。\n\n复现过程踩到的坑也记着。第一版自己给每道题写任务文件,27B 按它的计划做得更差;换成作者原始题面才对齐。托管方也不一样:作者用自建 vLLM 跑 FP8,复现走 OpenRouter 轮询 Modal、CoreWeave、Wafer,203 次调用落在 Modal,三次运行中途换了主机。复现工作流本身也不是人手写的,是 Memdoor 里的 coder agent 照着方法描述自己生成的八个 YAML 文件,不指定模型。\n\n## 这套方法真正值钱的地方\n\n不是 27B 打败了旗舰。论文的机制分析给了个更实在的解释:状态存在磁盘上,token 上限就截不断它。Qwen3.8-27B 有八次单次调用在推理中彻底失控,烧完 250k token 的预算一个字代码都没吐出来,有一次在 abc399_e 上把同一句话重复了 7,743 次。而单次调用的 35 个空白格里,管理者解回 28 个、失败 7 个,值 5.6 分。把每次调用切碎、让中间结论落盘,是在跟推理失控这个失效模式硬碰。\n\n还有一条更要紧:这套配方是纯提示词、纯文件、可移植。复现者能在完全不同的引擎上把它跑通,恰恰说明它不是某个框架的专利。\n\n## 所以呢\n\n如果编排已经是一条独立于模型的工程轴,那要不要换成更大的模型就不再是唯一问题,同一批预算可以先拿去改工作流。但这条轴有边界:Qwen3.6-35B 那种负收益是真的,账单翻三倍是真的,而两者都只在一道编程竞赛基准上试过,数学和知识类基准因接近满分被放弃。\n\n下次再看到小模型组队追平旗舰这种标题,先问两句:编排的钱算进去了吗?评测之外的那部分工作,也是这样编排的吗?","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.26480","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"e82b2d09-81b2-43d1-977e-e018443b3c14","coding-agent",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4111b224-0090-4e40-aaff-ba212b58601c","en","Five Qwen3.8-27B instances match Fable 5 with a file ledger","Ledger orchestration lifts five Qwen3.8-27B from 66.8% to 92.4% on LiveCodeBench-hard, past Fable 5's 90.4%. An outside rerun scored 9.8 vs 9.6.","\u002F\u002F Ledger-based orchestration lets five small Qwen3.8-27B instances beat Fable 5 on hard LiveCodeBench; an independent rerun confirms parity, not a win.\n\nA coding harness that went viral on Hacker News claims five Qwen3.8-27B instances, coordinating through files on disk, push hard LiveCodeBench pass@1 from 66.8% to 92.4%, ahead of Claude Fable 5's single-call 90.4%. On October 31 an outside developer rebuilt it in a different tool, published all 50 runs, and reached a more cautious verdict: parity, not superiority.\n\n## What the method actually does\n\nThe paper calls it ledger-based zero-shot self-orchestration, GVS5H, which the authors expand as Gorilla vs. 5 Humans. Nothing is trained. Every role is a fresh call of the same model, and the roles never talk directly; they exchange information only through files. The problem sits in task.md, the manager writes three to six sentences of strategy into plan.md, ideas and partial arguments go into notes.md, and the current attempt is solution.py. Each round the manager assigns exactly one task; the worker rewrites both files, and a shell script running the public samples decides success. The loop runs at most 10 rounds, with plan.md capped at 4,000 characters and notes.md at 8,000.\n\nEvaluation uses the 100 hardest LiveCodeBench release_v6 problems by contest date, scored with a corrected evaluator. The paper's own numbers: Qwen3.8-27B goes from 66.8% single-call to 92.4% with the manager, a 25.6 point gain; Qwen3.8-Flash-Next moves from 84.2% to 93.0% at 9% of the cost; DeepSeek-V4.1-Flash from 88.4% to 92.0%; GPT-5.6-Terra from 80.8% to 88.0% and GPT-5.6-Luna from 70.4% to 81.2%. The headline condition runs six models five times each, five of them with both a single-call and an orchestrated arm, while Fable 5 was run single-only; five further models went through the OpenRouter gateway for a single pass.\n\n## What the rerun produced\n\nOn October 31 the author of Memdoor rebuilt the method in his own, different workflow engine: the same Qwen3.8-27B, the same prompts, the same 128K reply cap, the same 10-round limit, and every message sent to the model matched the original byte for byte, checked by a parity script where a one-character change makes the two sides differ. He tested the authors' first 10 problems over five passes.\n\nResults: the rebuilt workflow averaged 9.8, the authors' own harness 9.6, Fable 5 alone 8.0. 49 of 50 runs solved the problem, emitting 15.3M output tokens for $35.79, or $38.77 including the 16 attempts cut off by a clock, stopped for a fix, or paused. A run took a median of 40 minutes of model time and the longest ran four hours, all on OpenRouter rather than a local machine.\n\n## Three things the paper admits\n\nFirst, the gains are not universal. The manager is not free, the paper says: its deliberation can settle on a worse answer than a single call would have produced. Qwen3.6-35B with reasoning off is negative at both caps, because its ideation stage identified the same optimization the single call had used, talked itself out of it as too error-prone, and committed to an asymptotically slower plan that a worker then implemented with a further bug. The LCB 3765 problem shows the pattern in full.\n\nSecond, Fable 5 was run single-only because it is expensive, so how much its manager arm would gain is unknown. It also returned refusals on 6 of 500 problem-passes, which the protocol scores as failures rather than substituting a fallback model.\n\nThird, the bill roughly triples. Qwen3.8-27B single-call costs $18.95 per pass and $0.28 per solved problem; with the manager it becomes $47.82 and $0.52. Fable 5 single-call is $61.11. The conclusion that orchestration is often a cheap substitute for model scale carries that premise.\n\n## Four caveats the rerun author wrote down\n\nTen problems, not one hundred, and on a sample that small 9.8 and 9.6 are the same number, so the claim is parity with the authors' harness rather than a gain on it. The contests date from 29 March to 6 April 2025 and the model may have seen them in training, so contamination cannot be ruled out. Fable 5's column comes from the authors' result files and was not run in this harness. And the loop sees only the public samples while the score comes from hidden tests.\n\nThat last asymmetry is exactly what Hacker News objected to most: comparing an orchestrated Qwen against an un-orchestrated Fable 5. The rerun author simply inherited it.\n\nThe gaps he hit are recorded too. The first version wrote its own task file per problem and the 27B planned worse from it than from the authors' exact LiveCodeBench text; the host also differed, since the authors served their own FP8 vLLM while the rerun rotated across Modal, CoreWeave and Wafer on OpenRouter, with 203 calls served by Modal and three runs switched mid-run. The workflow files were not hand-written at all: an agent inside Memdoor generated eight YAML files from a description of the method, naming no model.\n\n## Where the value actually sits\n\nNot in a 27B beating a frontier model. The paper's transcript analysis offers a more useful explanation: state lives on disk, so a token cap cannot truncate it. Eight Qwen3.8-27B single calls became unstable during reasoning and used up the full 250k budget without emitting code, one repeating the same phrase 7,743 times on abc399_e. Of the single arm's 35 empty cells, the manager solved 28 and failed 7, worth 5.6 points. Splitting each call and writing intermediate results to disk attacks that failure mode head on.\n\nMore important still, the recipe is prompts and files only, so it is portable. Running it on a completely different engine is what shows it is not one framework's patent.\n\n## So what\n\nIf orchestration is already an engineering axis independent of the model, whether to switch to a bigger one stops being the only question, and the same budget can go into the workflow first. But the paper and the rerun together say the axis has limits: the negative result on Qwen3.6-35B is real, the tripled bill is real, and both were tested on one coding-contest benchmark, with math and knowledge benchmarks abandoned at near-perfect saturation.\n\nNext time a headline claims small models teaming up to match a frontier one, ask two questions. Was the orchestration cost counted? And is the work outside the benchmark orchestrated the same way?","gvs5h-ledger-orchestration-qwen-fable-rerun","2026-11-01T09:20:28Z","2026-11-01T09:20:33.820187Z","2026-11-01T09:20:33.820199Z",true,"agent",27,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"0237222a-602b-47ef-9431-468009904428","FACET 先建环境再写任务:1.2K 轨迹把 Qwen3.5-27B 推到 Terminal-Bench 47.57,逼近 397B","facet-terminal-task-synthesis","2026-08-19T06:19:20+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"05fe5d2f-b83d-4aa9-9a68-5d0274b0d558","DeepSeek 开源 Super-agent:双层架构调度","deepseek-super-agent-router-worker-architecture","2026-11-01T07:31:50+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"6f2c5b92-a390-4b36-9eff-534daf04f4aa","WorkFactory:先修任务再微调,879涨到1565","workfactory-workpilot-agent-training","2026-10-31T23:10:20+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"9109e6b0-bf85-472d-abf6-ba8e31dc17c5","Co-Design 基准:七成准则过,26.4% 完整交付","co-design-bench-agent-delivery-gap","2026-10-30T05:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"dcc6132a-e432-4ef3-9555-5412d692cd90","RSIHub 开源:先冻结评测器,再谈自我进化","rsihub-agent-evolution-frozen-evaluator","2026-10-26T07:20:28+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"9597f154-5515-457e-9590-8b32347ea750","Nanbeige4.3-33B:33B教师模型自测80.9分","nanbeige-4-3-33b-open-agent-teacher-model","2026-10-26T01:15:00+00:00"]