// Ledger-based orchestration lets five small Qwen3.8-27B instances beat Fable 5 on hard LiveCodeBench; an independent rerun confirms parity, not a win.

A coding harness that went viral on Hacker News claims five Qwen3.8-27B instances, coordinating through files on disk, push hard LiveCodeBench pass@1 from 66.8% to 92.4%, ahead of Claude Fable 5's single-call 90.4%. On October 31 an outside developer rebuilt it in a different tool, published all 50 runs, and reached a more cautious verdict: parity, not superiority.

What the method actually does

The paper calls it ledger-based zero-shot self-orchestration, GVS5H, which the authors expand as Gorilla vs. 5 Humans. Nothing is trained. Every role is a fresh call of the same model, and the roles never talk directly; they exchange information only through files. The problem sits in task.md, the manager writes three to six sentences of strategy into plan.md, ideas and partial arguments go into notes.md, and the current attempt is solution.py. Each round the manager assigns exactly one task; the worker rewrites both files, and a shell script running the public samples decides success. The loop runs at most 10 rounds, with plan.md capped at 4,000 characters and notes.md at 8,000.

Evaluation uses the 100 hardest LiveCodeBench release_v6 problems by contest date, scored with a corrected evaluator. The paper's own numbers: Qwen3.8-27B goes from 66.8% single-call to 92.4% with the manager, a 25.6 point gain; Qwen3.8-Flash-Next moves from 84.2% to 93.0% at 9% of the cost; DeepSeek-V4.1-Flash from 88.4% to 92.0%; GPT-5.6-Terra from 80.8% to 88.0% and GPT-5.6-Luna from 70.4% to 81.2%. The headline condition runs six models five times each, five of them with both a single-call and an orchestrated arm, while Fable 5 was run single-only; five further models went through the OpenRouter gateway for a single pass.

What the rerun produced

On October 31 the author of Memdoor rebuilt the method in his own, different workflow engine: the same Qwen3.8-27B, the same prompts, the same 128K reply cap, the same 10-round limit, and every message sent to the model matched the original byte for byte, checked by a parity script where a one-character change makes the two sides differ. He tested the authors' first 10 problems over five passes.

Results: the rebuilt workflow averaged 9.8, the authors' own harness 9.6, Fable 5 alone 8.0. 49 of 50 runs solved the problem, emitting 15.3M output tokens for $35.79, or $38.77 including the 16 attempts cut off by a clock, stopped for a fix, or paused. A run took a median of 40 minutes of model time and the longest ran four hours, all on OpenRouter rather than a local machine.

Three things the paper admits

First, the gains are not universal. The manager is not free, the paper says: its deliberation can settle on a worse answer than a single call would have produced. Qwen3.6-35B with reasoning off is negative at both caps, because its ideation stage identified the same optimization the single call had used, talked itself out of it as too error-prone, and committed to an asymptotically slower plan that a worker then implemented with a further bug. The LCB 3765 problem shows the pattern in full.

Second, Fable 5 was run single-only because it is expensive, so how much its manager arm would gain is unknown. It also returned refusals on 6 of 500 problem-passes, which the protocol scores as failures rather than substituting a fallback model.

Third, the bill roughly triples. Qwen3.8-27B single-call costs $18.95 per pass and $0.28 per solved problem; with the manager it becomes $47.82 and $0.52. Fable 5 single-call is $61.11. The conclusion that orchestration is often a cheap substitute for model scale carries that premise.

Four caveats the rerun author wrote down

Ten problems, not one hundred, and on a sample that small 9.8 and 9.6 are the same number, so the claim is parity with the authors' harness rather than a gain on it. The contests date from 29 March to 6 April 2025 and the model may have seen them in training, so contamination cannot be ruled out. Fable 5's column comes from the authors' result files and was not run in this harness. And the loop sees only the public samples while the score comes from hidden tests.

That last asymmetry is exactly what Hacker News objected to most: comparing an orchestrated Qwen against an un-orchestrated Fable 5. The rerun author simply inherited it.

The gaps he hit are recorded too. The first version wrote its own task file per problem and the 27B planned worse from it than from the authors' exact LiveCodeBench text; the host also differed, since the authors served their own FP8 vLLM while the rerun rotated across Modal, CoreWeave and Wafer on OpenRouter, with 203 calls served by Modal and three runs switched mid-run. The workflow files were not hand-written at all: an agent inside Memdoor generated eight YAML files from a description of the method, naming no model.

Where the value actually sits

Not in a 27B beating a frontier model. The paper's transcript analysis offers a more useful explanation: state lives on disk, so a token cap cannot truncate it. Eight Qwen3.8-27B single calls became unstable during reasoning and used up the full 250k budget without emitting code, one repeating the same phrase 7,743 times on abc399_e. Of the single arm's 35 empty cells, the manager solved 28 and failed 7, worth 5.6 points. Splitting each call and writing intermediate results to disk attacks that failure mode head on.

More important still, the recipe is prompts and files only, so it is portable. Running it on a completely different engine is what shows it is not one framework's patent.

So what

If orchestration is already an engineering axis independent of the model, whether to switch to a bigger one stops being the only question, and the same budget can go into the workflow first. But the paper and the rerun together say the axis has limits: the negative result on Qwen3.6-35B is real, the tripled bill is real, and both were tested on one coding-contest benchmark, with math and knowledge benchmarks abandoned at near-perfect saturation.

Next time a headline claims small models teaming up to match a frontier one, ask two questions. Was the orchestration cost counted? And is the work outside the benchmark orchestrated the same way?