Gel bands, colony plates, z-stacks of microscopy — this is the evidence wet-lab scientists wrestle with every day. Can AI agents read it correctly on their behalf? OpenAI's answer is blunt: across a hundred routine wet-lab tasks, its top-scoring model, GPT-6 Astra, earns on average only about two-thirds of the available credit — and still miscounts colonies.
100 tasks, all from real experiments
BioBasicsBench was built with Ligo, a biotech company focused on enzyme design. It covers five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. The tasks were not lifted from textbooks: scientists turned data from their own experiments — including failed runs and ambiguous images — into questions, each paired with an expert-written grading rubric and put through three reviews: a Ligo scientist who had not written the task, an external PhD-level scientist working in a wet lab, and a final internal review. Answers are scored against individual rubric criteria, so the benchmark can tell a correct conclusion apart from a partially correct analysis.
The evidence stays deliberately messy. Gels and blots appear in 31 tasks; protocols, notes and specs in 27; tables and instrument exports (including raw flow-cytometry FCS files) in 20; bench, apparatus and reagent photos in 18; microscopy in 14; instrument screens and plots in 13; sequencing files in 12; protocol videos and agar plates in 7 each; and 21 tasks carry reference sheets such as ladder charts and plasmid maps (categories overlap). OpenAI says existing benchmarks built around figures selected for papers miss exactly this kind of ambiguity and troubleshooting.
Seven models queue up; the top score is 66.7%
OpenAI evaluated seven models — from GPT-5.2 to GPT-6 Astra — across 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting: 18,500 graded results in total. Mean reward at the highest xhigh effort:
| Model | Mean reward | Output tokens / task |
|---|---|---|
| GPT-6 Astra | 66.7% | 4,704 |
| GPT-6.1 Sol | 62.0% | 5,238 |
| GPT-5.6 Sol | 43.8% | 9,099 |
| GPT-6 Luna | 38.3% | 13,120 |
| GPT-5.5 | 35.4% | 11,135 |
| GPT-5.4 | 33.2% | 13,717 |
| GPT-5.2 | 31.2% | 27,620 |
Two signals stand out. First, an 18.2-point cliff separates GPT-6.1 Sol from GPT-5.6 Sol; OpenAI reads this as stronger native multimodal capability being key (the vendor's own interpretation). Second, efficiency flips between generations: GPT-5.2 burns 27,620 output tokens per task for 31.2% — roughly 5.9x what Astra spends — so the new generation is not winning by writing longer reasoning chains.
The top model still faceplants in the lab
OpenAI showcases three failures from its own leader. In a colony PCR task, Astra grouped the gel bands correctly but reversed the experimental attribution: assuming the larger PCR product marked successful cloning, it picked lanes 4, 6, 7 and 8, while the reference — following the controls — selects lanes 5, 9 and 10. Follow its pick and you would purify the wrong plasmid downstream. In a counting task its tally runs about 12% over the reference: close neighbours merged into one mark, visible colonies left unmarked, empty agar counted as a colony — and an inflated count inflates the transformation-efficiency estimate by the same proportion at fixed plated volume and dilution. In another task it inspected both darker and lighter vacuoles, then kept only the half of the evidence that supported its 'vacuoles are darker' conclusion.
There are genuine wins too. In a DNA-cloning task, GPT-5.6 Sol saw no colonies on the negative-control plate and concluded away; Astra noticed the rim label read LB Kan50 — kanamycin — while the experiment used ampicillin, and correctly rejected that plate as a reliable control. In a dead-cell staining task, GPT-5.6 Sol called the cells dead from a single dye's brightness, while Astra pointed out that the two detection channels overlap at 525–540 nm, that brightness alone cannot settle the question, and asked for better controls. In another task it recognized cells 'disappearing' as a z-stack depth scan rather than a biological event.
So what?
The benchmark separates answering biology questions from doing experiments: the latter is about resisting over-interpretation and admitting when the evidence is insufficient. 66.7% means no model can yet be trusted with routine wet-lab analysis — the upper bound of Astra's 95% confidence interval is 73.1%. OpenAI closes by saying that tracking these capabilities will also help us understand their implications for biosafety and biosecurity — this leaderboard is both a capability chart and a risk register. For anyone building AI for science, don't ask whether the model can enter the lab; ask whether it dares to say: this gel does not support a conclusion. (Source: OpenAI's announcement)