There is no universal teacher or definition of a good AI answer. The people responsible for a system’s particular use must decide what success means, drawing on domain experts, evaluators, users, and others affected by its answers. Teams then turn those expectations into examples, scoring criteria, and ongoing checks. A benchmark score can help, but it cannot by itself establish that an AI works well for every person or situation.
Who decides what “good” means?
It depends on the task and its consequences. A concise answer may be best for a simple lookup; a health response may need to be accurate, acknowledge uncertainty, and make clear when a person should seek professional care. A workplace assistant may need to follow a specified format as well as get the substance right.
The team accountable for the AI’s use sets the objective and constraints. But it should not assume that developers alone can define quality. Domain experts can identify what competent work requires; evaluators can make criteria consistent; users can reveal whether answers meet their needs; and people affected by the system can surface risks that its builders may overlook. Their roles and influence will vary by application. There is no single agreed rubric that fits every domain.
How an idea of quality becomes something testable
A useful evaluation starts by stating the outcome, not by choosing a convenient score. OpenAI’s evaluation guidance frames the first question as: “What’s the success criteria for the eval?” It then recommends selecting a dataset and metrics, comparing results, iterating, and continuing to evaluate as a system changes. These are planning steps, not universal pass thresholds. OpenAI’s API evaluation guidance also describes a good answer in terms of precise answers, appropriate use of context, and meeting the user’s need.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Define the intended outcome. Specify what the AI is supposed to help someone do, and what it must not do. “Answer questions” is too broad to evaluate reliably.
- Set assessable criteria. Translate the outcome into observable expectations, such as whether an answer is factually supported, uses relevant context, follows required instructions, communicates uncertainty appropriately, or avoids a specified unsafe action. Include only dimensions that matter to the use.
- Involve people who know the work and its impact. Ask domain practitioners what a competent answer looks like, and seek user and affected-person perspectives where appropriate. Decide who will resolve disagreements about borderline cases.
- Build representative examples. Include ordinary prompts as well as difficult, ambiguous, and edge cases. Examples should reflect the questions and contexts the system is meant to handle.
- Choose checks that fit the criteria. Some requirements can be checked automatically; others need qualified human judgment or tests in realistic settings. Compare the system with a baseline or earlier version when that helps answer the evaluation question.
- Re-evaluate as the system changes. Changes to the model, prompts, tools, users, or operating context can alter performance. Keep evaluation connected to those changes rather than treating an initial score as permanent proof.
A rubric makes judgment more inspectable: it records what evaluators should look for and how they should distinguish stronger from weaker answers. It does not make a subjective or contested judgment automatically objective. Teams still need to check that the rubric reflects the intended task and that evaluators apply it consistently.
What domain-specific evaluation looks like
Health answers: criteria built with physicians
OpenAI’s HealthBench illustrates how evaluation can be tailored to a high-stakes domain. Its benchmark was developed with 262 physicians who had practice experience in 60 countries. It contains 5,000 realistic health conversations, each paired with a physician-created rubric, and uses 48,562 unique rubric criteria for model-based grading. These figures describe the design of this particular benchmark; they do not show that every criterion represents universal medical consensus or that every grade has been independently validated.
Workplace tasks: compare deliverables against expert judgment
In OpenAI’s GDPval, task writers created detailed rubrics for occupational tasks, and experienced professionals blindly compared and ranked work produced by models and humans. Its gold set contains 220 tasks. OpenAI says its experimental automated grader estimates expert judgments; it is not a replacement for the expert graders. The example shows why defining quality can require people who understand both the task and what counts as competent work in that occupation.
What a benchmark score does—and does not—tell you
A benchmark score answers a bounded question about performance on a specified set of tests under specified conditions. It is not the same as an estimate of how well a system will perform across the wider population of similar questions. NIST distinguishes fixed-benchmark accuracy from generalized accuracy: the latter asks how performance extends beyond the benchmark to questions drawn from a broader target population. NIST’s February 19, 2026 explanation of AI evaluation and measurement stresses that there is no one-size-fits-all formula for quantifying performance.
Free tools Windows power users keep installed
One-click scans. No signup required.
Before relying on a result, check what it actually measures: the task, examples, scoring method, and conditions. Ask whether the examples resemble real use, whether the rubric covers the important kinds of quality, and whether a small score difference is meaningful. A strong result on a narrow test can be useful evidence without proving broad reliability, safety, or user satisfaction.
Why evaluation can be misleading
The test itself can introduce errors. Anthropic reported that simple formatting changes produced about a 5% change in MMLU accuracy in its tests. That is an observation about those tests, not a general effect size for all benchmarks. Anthropic also identifies risks including training exposure to benchmark questions, inconsistent implementations, and test items that are erroneous or unanswerable. Its discussion of challenges in evaluating AI systems underscores why the benchmark’s implementation and limitations matter alongside the score.
- Benchmark contamination: If test material appeared in training data, a score may not reflect performance on genuinely new questions.
- Implementation differences: Prompt formatting, scoring code, and other setup choices can affect results and make comparisons less reliable.
- Weak or ambiguous items: An evaluation can reward the wrong behavior if a question has no sound answer or its intended answer is unclear.
- Overly narrow criteria: A system may score well on what is easy to count while missing qualities that matter to users in practice.
Automated graders also need scrutiny. They can make evaluation more scalable, but their judgments should be checked against the task’s requirements and, where the stakes warrant it, human assessments. Treat a grader as a measurement tool with its own possible errors, not as a neutral authority.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.When automated scores are not enough
Automated benchmarks are only one evaluation method. NIST’s January 2026 initial public draft, AI 800-2, is scoped to automated benchmark evaluations; it is a draft, not final guidance, and it notes that some evaluation objectives require other methods. Depending on the question, complementary approaches can include red teaming, human-subject experiments, field testing, and post-deployment monitoring. NIST AI 800-2 sets out that scope.
For AI agents that retrieve and use information, NIST describes probes that check answers against a human-curated document corpus and retain evidence trails. Those probes examine whether an answer is grounded in the corpus and whether its support is faithful, complete, and sufficient. The aim is to let reviewers inspect what the system found and how that evidence supports its conclusions, rather than accept an answer solely because the AI produced it. NIST’s agentic-AI probe work describes this approach.
How to judge an evaluation before trusting its result
- Task fit: Does the test measure the outcome the system is meant to deliver?
- Criteria ownership: Who wrote and validated the rubric, and did they understand the task and its effects on people?
- Measurement target: Is the result for a fixed benchmark, or is it intended to estimate performance across broader real-world questions?
- Coverage: Does the method check the relevant dimensions—such as factuality, context use, communication, safety, or robustness—rather than just what is easiest to score?
- Reliability: Are the procedure and scoring repeatable? Could contamination, formatting choices, ambiguous items, or grader errors change the result?
- Complementary evidence: Do the stakes or use context call for human review, red teaming, field tests, or monitoring in addition to automated scores?
The answer to “who teaches AI what a good answer is?” is therefore not one person, benchmark, or grading model. People define the intended quality for a particular use; teams encode it in criteria and examples; and evaluation checks whether the system meets those expectations within stated limits. The more consequential the task, the more important it is to make those judgments visible and test them with methods suited to the real use.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
