Five fresh-context Qwen3.8-27B agents, one shared disk, and a shell script that decides who is right: that is the whole trick behind Memdoor's team-solve workflow, which this week posted a mean of 9.8 out of 10 on the first ten LiveCodeBench Hard problems. Claude Fable 5, answering the same ten in a single call, posted 8.0. The numbers come from the project's GitHub repository, with all fifty runs published in full.
A replication that scored above the original
What makes the result unusual is that it is a replication, not a self-run benchmark. Memdoor rebuilt the GVS5H method by Lee (2026) — using the original authors' exact prompts and loop, on the first ten problems of their list, five passes each. The rebuilt harness averaged 9.8 of 10, ahead of the 9.6 the original authors measured for their own harness, and well clear of Fable 5's 8.0 in a single call. Across the full 100-problem list the original authors report 92.4 versus 90.4, so the gap narrows with scale — but the direction holds. Every one of the 50 runs is published: each request, reply, program and grader output can be inspected line by line.
Five roles, zero shared context
The mechanics matter more than the score. Five Qwen3.8-27B instances play planner, two parallel ideators, solver and manager, with a repair worker on top. Each role starts from a fresh context; the only channel between them is files on disk. The planner writes a plan file, the solver writes solution.py, the manager reads the test logs, the repair worker patches. A shell script that runs the samples decides success — not a model grading itself, so no agent gets to say "I think this is correct."
$35.79, 15.3M tokens and one contamination caveat
The economics are published too: fifty runs cost $35.79 — counting 16 incomplete attempts, $38.77 — and consumed 15.3 million output tokens; a median run took forty minutes and $0.39 on Qwen3.8-27B with a 128K reply cap. The authors flag contamination up front: the contests date from 29 March to 6 April 2025, and the model may have seen them in training, so contamination cannot be ruled out. The workflow itself is written in YAML and names no model — point it at an API model, or run it on local open weights through Ollama, llama.cpp, vLLM or LM Studio, with no key required.
Orchestration versus one big model
My read: this is not a victory lap for 27B models over the frontier. It is one more clean data point in the orchestration-versus-scaling argument. Stretched to the full hundred problems, 92.4 against 90.4 says the frontier model is not actually beaten; the forty-minute median run time is the quiet cost that multi-agent setups keep ignoring. What is genuinely valuable is the discipline: cold starts plus file-only communication rule out cross-contamination between agents, and publishing all fifty runs turns "replicable benchmark" from slogan into auditable engineering. Teams building agent pipelines should read the team-solve example and the Memdoor repository as a reference implementation. The next question almost asks itself: with the same discipline, how far can the model shrink — 14B, 7B — before the score falls apart?