iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
BioMysteryBench is Anthropic’s evaluation of Claude on open-ended bioinformatics problems that require analyzing real or minimally processed datasets, using appropriate tools and databases, and producing answers that can be checked against objective biological evidence. It is designed to test scientific investigation rather than isolated knowledge recall.
Anthropic’s April 29, 2026 report says the benchmark contains 99 expert-authored questions. Its results show a sharp gap between tasks at least one human expert solved and tasks an expert panel did not solve, with Claude’s answers on the latter group also proving less repeatable across five attempts.
What BioMysteryBench is designed to measure
Many language-model benchmarks present a self-contained question and grade a short answer. BioMysteryBench instead gives a model a scientific mystery embedded in data. The model may need to inspect sequence files, identify a suitable computational method, query a biological database, combine several clues and then explain a conclusion. The benchmark is intended to resemble the messy analytical work performed in bioinformatics laboratories.
Anthropic describes the benchmark as method-agnostic. A trajectory is graded on its final answer, so two systems can receive credit even if they take different analytical routes. One model might begin with sequence alignment while another starts with metadata or a database search; both can succeed if they reach the objectively supported result.
#1 Best Overall
The target answers are tied to measurable properties of the data or to metadata validated independently of the model’s interpretation. This distinction matters. The benchmark is not asking Claude to imitate a particular scientist’s narrative. It asks for a result that can be checked against a crystal structure, a validated sample identity, an expression signature or another external biological fact.
Examples of benchmark questions
- Identify an organism from a protein crystal structure.
- Identify a virus from RNA sequencing data when sample metadata has been validated by PCR.
- Determine which gene was knocked out in experimental samples compared with controls using RNA-seq data.
- Infer relationships among samples from whole-genome sequences.
- Answer questions such as which human organ produced a single-cell RNA-sequencing dataset.
These are investigative questions: the answer is not necessarily written in a prompt, and the solver must extract evidence from the supplied data.
Data types and scientific coverage
Most BioMysteryBench tasks use raw or minimally processed DNA or RNA sequencing data. The collection also reaches beyond sequencing, covering several major forms of computational biology.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →| Data or field | What the benchmark includes |
|---|---|
| Whole-genome sequencing | Questions about sequence-based identity and relationships among samples |
| Single-cell RNA sequencing | Cell-type and tissue or organ identification problems |
| Methylation | Tasks using DNA-methylation signals |
| ChIP-seq | Questions based on protein-DNA binding or chromatin-associated sequencing data |
| Metagenomics | Analysis of mixed microbial communities |
| Hi-C | Chromosome-conformation and three-dimensional genome data |
| Proteomics | Protein-level measurements and identification tasks |
| Metabolomics | Small-molecule and metabolic-profile analysis |
Each question author supplied a validation notebook showing that the relevant signal exists in the data. That quality check helps establish that a task is well formed, but it does not guarantee that every solver will locate the signal or that the problem will be easy in practice.
How the benchmark establishes a human baseline
Anthropic asked up to five domain experts to solve each question from scratch. A task counted as human-solvable when at least one expert answered correctly. On that basis, Anthropic reports 76 human-solvable tasks.
The report also identifies 23 human-difficult tasks after four broken or malformed tasks were removed during quality review. “Human-difficult” describes the outcome for this particular expert panel. It does not prove that no human anywhere could solve those questions, nor that the questions are impossible.
| Reported category or event | Figure | How to interpret it |
|---|---|---|
| Expert-authored questions | 99 | The size of the benchmark reported by Anthropic in 2026 |
| Human-solvable tasks | 76 | At least one of up to five experts answered correctly |
| Human-difficult tasks | 23 | Tasks the panel did not solve, reported after four malformed tasks were removed |
| Quality-control removals | 4 | Broken or malformed tasks excluded from the difficult-task analysis |
This baseline is useful because it separates ordinary scientific difficulty from model performance. A score on the human-solvable group indicates whether a model can handle problems that at least one expert completed. A score on the human-difficult group probes a more demanding frontier, but the label carries more uncertainty because the benchmark cannot tell whether an unsolved task is exceptionally hard or genuinely impossible.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteAnthropic’s reported Claude results
Anthropic says Claude Sonnet 4.6 and more capable models solved meaningful fractions of the human-difficult set. The report gives one specific headline result: Claude Mythos Preview solved 30% of those human-difficult tasks.
For Claude Opus 4.6, the contrast between the two groups is explicit. It achieved 77.4% on human-solvable tasks and 23.5% on human-difficult tasks. These percentages describe different difficulty populations and should not be read as a single overall accuracy rate.
| Model or measurement | Task set | Reported result |
|---|---|---|
| Claude Opus 4.6 | Human-solvable tasks | 77.4% |
| Claude Opus 4.6 | Human-difficult tasks | 23.5% |
| Claude Mythos Preview | Human-difficult tasks | 30% solve rate |
These are Anthropic’s benchmark findings, not an independent replication. The report does not establish that Claude will achieve the same percentages on a different collection of organisms, laboratories, sequencing platforms or questions.
Why repeatability changes the meaning of a score
Anthropic evaluated five attempts per problem rather than counting only one run. That exposes a weakness hidden by a conventional accuracy percentage: a model may reach the correct answer once through a lucky chain of tool choices and fail on its next attempt.
For Opus 4.6, 86% of the problems it solved in the human-solvable set were solved at least four times out of five. On the human-difficult set, the corresponding share was 44%. The report also states that one- or two-of-five wins rose from 9% on human-solvable problems to 44% on human-difficult problems.
Rank #3
| Opus 4.6 reliability measure | Human-solvable set | Human-difficult set |
|---|---|---|
| Problems solved at least four times out of five, among problems it solved | 86% | 44% |
| Wins occurring only once or twice in five attempts | 9% | 44% |
The practical reading is straightforward: the difficult-set solve rate is more brittle. A single successful transcript may demonstrate capability, but repeated success is stronger evidence that the model has a dependable analytical strategy. For scientific use, teams should therefore record multiple runs, preserve intermediate files and compare whether the same evidence supports the answer each time.
What appears to help Claude on difficult tasks
Anthropic’s transcript analysis identifies two recurring patterns in successful attempts.
Connecting prior knowledge to biological databases
Claude can use background knowledge to link a biological mechanism with a candidate organism, pathway or database record. That connection can narrow a large search space faster than treating every datum as unrelated. However, prior knowledge is not automatically reliable. Anthropic describes at least one example in which it misled the model, so a plausible remembered association must be checked against the supplied data.
Combining several analytical methods
When one signal is ambiguous, successful attempts often combine methods rather than relying on a single diagnostic. A model may compare sequence evidence with metadata, expression patterns with known gene function, or structural clues with database annotations. This flexibility is one reason a method-agnostic benchmark can reward different trajectories.
Neither pattern removes the need for verification. A database hit can be wrong because of a close homolog, contamination or incomplete metadata, and a biologically plausible explanation can still contradict the measured signal.
How a BioMysteryBench task is evaluated
The benchmark’s design can be understood as a sequence of responsibilities for the solver and the grader:
Rank #4
- Receive a scientific question and data. The task may include raw or minimally processed sequencing data, other omics measurements and sample metadata.
- Choose an analytical route. The model can select tools, databases and computational methods instead of following a fixed script.
- Extract and combine evidence. It may need to identify quality signals, compare samples, search reference information or connect multiple biological clues.
- State an answer that has an objective check. The expected result is grounded in a data property or independently validated metadata.
- Grade the final answer. Anthropic evaluates the conclusion rather than requiring one prescribed trajectory.
- Repeat the attempt. Five attempts reveal whether a reported win is stable or occasional.
This workflow differs from a demonstration in which a model is shown the correct analysis path. BioMysteryBench leaves the route open, which makes tool selection and error recovery part of the challenge.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →How BioMysteryBench differs from other scientific evaluations
Anthropic positions BioMysteryBench between several common evaluation styles. Knowledge and reasoning tests are clean and easy to score, but they may not capture the difficulty of working with noisy files. Benchmarks that compare a model with a published scientist’s conclusion can measure agreement, yet agreement is not always the same as independently verifiable correctness. Simulated laboratory environments provide controlled ground truth, but their data and actions may be less representative of routine bioinformatics.
BioMysteryBench emphasizes five comparison questions when assessing a scientific benchmark:
- Are the data real and noisy, or simulated and synthetic?
- Is there objective ground truth, or is the answer judged mainly against a scientist’s conclusion?
- Can the model choose tools and analytical methods?
- Does the set include questions that the human panel did not solve?
- Does the evaluation report repeated-attempt reliability as well as aggregate accuracy?
Its stated goal is to combine realistic data complexity with answer verification. Anthropic summarizes that design as using “messy, real-world bioinformatics data, without allowing the complexity and challenges inherent in this data to corrupt the quality of the evaluation.”
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.CompBioBench context: related, not a head-to-head comparison
Anthropic’s post also mentions CompBioBench, released by Genentech and Roche while the BioMysteryBench article was being finalized. It describes 100 computational-biology tasks built from synthetic or augmented data with scrubbed metadata. Anthropic reports Claude Opus 4.6 at 81% overall and 69% on the hardest questions on that benchmark.
Free tools Windows power users keep installed
One-click scans. No signup required.
| Benchmark | Task count | Data description | Reported Claude Opus 4.6 result |
|---|---|---|---|
| BioMysteryBench | 99 questions | Mostly raw or minimally processed real-world DNA and RNA data, plus proteomics and metabolomics | 77.4% on human-solvable tasks; 23.5% on human-difficult tasks |
| CompBioBench | 100 tasks | Synthetic or augmented data with scrubbed metadata | 81% overall; 69% on hardest questions |
Those figures should not be treated as a leaderboard. The datasets, task construction, difficulty definitions and scoring conditions differ, so the numbers do not constitute a controlled BioMysteryBench-versus-CompBioBench experiment.
Best Value
Limitations that affect interpretation
The human panel is not a proof of impossibility
A task becomes human-difficult because up to five recruited experts did not solve it. A larger or differently specialized group might perform differently. The category is therefore a useful reference point, not a universal boundary between solvable and unsolvable science.
Validation shows signal, not discoverability
A validation notebook can demonstrate that a biological signal exists and that the data are well formed. It cannot guarantee that a solver will find the signal, select the right tool, avoid a confounder or recognize the relevant database entry. Unsolved tasks remain ambiguous for that reason.
Hard-task wins are less reproducible
The five-attempt analysis shows that difficult-set wins are often one-off successes. A headline solve rate without a repeatability measure can therefore overstate how dependable the capability is.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteReported results come from the benchmark owner
The percentages and reliability figures are Anthropic’s own analysis of its benchmark. They are valuable evidence about the tested set, but they are not an independent replication across models, laboratories or task authors.
What the results mean for scientific users
BioMysteryBench supports a narrower conclusion than “Claude can do bioinformatics.” It shows that newer Claude systems can sometimes complete open-ended, data-driven biological investigations, including some questions that the tested human panel did not answer. It also shows that success is uneven and less repeatable on those difficult questions.
For a real project, a responsible workflow would treat a model’s answer as a hypothesis until the underlying evidence is independently checked. Run the analysis more than once, compare the data objects and database records cited by each run, and have a domain expert inspect the final interpretation when the result affects an experiment, clinical decision or publication. Agreement between repeated attempts is useful, but it is not a substitute for validating the biological claim against the source data.
The benchmark’s method-agnostic design is particularly relevant when evaluating an agent that can write code, call command-line tools or search specialist databases. A system that reaches the right answer by a different route should still receive credit, while reproducibility checks reveal whether that route can be trusted beyond a single lucky attempt.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Bottom line on evaluating Claude with BioMysteryBench
BioMysteryBench is a serious test of scientific data analysis rather than a trivia exam. Its 99 questions span sequencing and other omics fields, use answers tied to objective biological evidence, and permit multiple analytical approaches. Anthropic reports strong performance on tasks at least one expert solved, substantially lower performance on the panel’s human-difficult tasks, and markedly weaker repeatability in that harder group. The fairest conclusion is that Claude demonstrates promising but still variable bioinformatics problem-solving ability; the benchmark does not establish general scientific superiority or guarantee reliable results on unseen datasets.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
