OpenAI introduced BioBasicsBench on November 6, 2026, a 100-task benchmark developed with enzyme-design company Ligo to test whether AI agents can interpret raw evidence from routine wet-lab experiments.
OpenAI reports results as mean reward, the average task score expressed as a percentage of available credit. At xhigh, the highest reasoning-effort setting reported for all seven models tested, GPT-6 Astra reached 66.7% with a 95% confidence interval of 60.3–73.1%, while GPT-5.6 Sol scored 43.8% with an interval of 37.6–50.0%. Astra averaged 4,704 output tokens per task, compared with 9,099 for Sol.
Built From Routine Experimental Evidence
BioBasicsBench addresses a limitation OpenAI identifies in evaluations built around charts and images selected for scientific papers: everyday lab work also involves failed experiments, ambiguous images, conflicting observations and troubleshooting. Its tasks ask agents to interpret experimental data, spot errors and recognize when the evidence cannot justify a conclusion. They span protein methods, imaging and cytometry, quantification and assays, nucleic acid methods and molecular cloning.
Scientists created the tasks from their own experiments and wrote grading rubrics. Inputs include gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports, bench photographs and protocol videos. Individual tasks can combine several evidence types and supporting reference sheets, such as ladder charts and plasmid maps. Agents return structured conclusions, measurements or recommendations, and rubrics award credit for elements including numerical results, supporting reasoning and recognition that the evidence is insufficient.
Each task received an internal review by a Ligo scientist who had not written it, an external review by a PhD-level wet-lab scientist and a final internal review covering the prompt, evidence, rubric, author notes and a graded model attempt. The evaluation covered 37 model and reasoning-effort settings, with five attempts on every task per setting, producing 18,500 results. Confidence intervals were calculated by resampling whole tasks while keeping each task’s five attempts together.
Examples Behind Astra’s Improvement
In a cell-imaging task, GPT-5.6 Sol classified cells as dead from a dye signal. Astra recognized that overlapping detection windows could allow interference from a second dye and called for controls that could separate the signals. In a DNA-cloning task, Sol based its conclusions on seeing no colonies on a negative-control plate, whereas Astra noticed that the plate was labeled for kanamycin rather than the ampicillin used in the experiment and rejected it as a reliable control.
Another task showed cells moving in and out of focus. Sol sought a biological explanation for their apparent disappearance, while Astra identified a z-stack, a series of images captured at different depths. OpenAI interprets these examples as evidence of better multimodal handling, use of experimental context and recognition of what data can and cannot establish, and as an indication that stronger native multimodal capabilities may be important for routine wet-lab analysis.
Where the Benchmark Exposes Failures
OpenAI cautions that higher scores do not yet amount to reliable laboratory judgment. In a colony-PCR task, Astra selected lanes 4, 6, 7 and 8 after assuming that a larger PCR product indicated successful cloning; the reference interpretation selected lanes 5, 9 and 10. That mistake would lead to purification of the incorrect plasmid.
Other examples show correct observations failing to govern the final answer. Astra generalized that vacuoles, compartments inside cells, are generally darker than surrounding cell material despite lighter vacuoles in the same image. Changing focal depth can reverse the contrast, and Astra’s own intermediate observations noted that contrast varied with focus. In a colony-counting task, OpenAI reports that Astra failed to separate close neighbors, missed visible colonies and marked empty agar, producing an approximately 12% overcount relative to the benchmark reference. With plated volume and dilution fixed, such an overcount inflates the estimated efficiency of DNA introduction by the same proportion.
OpenAI says following the development of these capabilities will also inform its assessment of their biosafety and biosecurity consequences.
The practical bar is therefore higher than a better aggregate score: an agent must carry accurate observations through to its conclusion, check controls and withhold conclusions the evidence cannot support. BioBasicsBench makes those requirements targets for improvement, and its worked failures show why progress toward laboratory assistance still depends on closing the gap between observing evidence and making a sound experimental decision.
Credit: Source link
