A demo shows that an AI feature can work. It does not show that it works on the thousand requests your users are about to send it. An eval (short for "evaluation") answers that question. It is a set of real test cases, an expected result for each one, and a score worked out the same way on every run.
A demo is not a measurement
An LLM does not always give the same answer to the same question. Change one sentence in the instructions, switch to a new version of the model, add a document: some answers get better, others get worse. Without a measurement, nobody knows which effect wins out.
Key concept
An eval is a repeatable test: the same cases, scored the same way, on every change. It replaces "looks good to me" with a score you can compare from one version to the next. Decide what "good enough" means before you build, not after the demo.
On a hundred cases, the feature scores 88% overall.
The test cases matter more than the tooling. They must look like what your users really send, hard cases included: an off-topic question, a document that is too long, a request even a person would find ambiguous. In October 2026, Anthropic's documentation advises favoring volume: plenty of automatically graded cases beat a handful graded by hand.
There are three ways to score. A script compares the output with the expected result; it is fast and reliable, but it can only judge what can be compared exactly. A person judges better, but is slow and expensive. A model can grade against a rubric, provided you have checked that its grades are right.
It all starts from a set of real test cases, with an expected result for each one.
What it means for you
For product managers
Write the success criteria before the first prototype, and make them measurable: "95% of refund requests sorted into the right category", not "quality answers". Start from twenty real cases, taken from tickets or emails, and have the person who handles those requests today write the expected result for each. A script or a model then scores every run. On twenty cases, each one is worth 5 points: enough to spot errors, not to prove a 95% threshold. In a startup, that first set is enough to decide whether to carry on. In a large company, it becomes the evidence you will be asked for before the feature is opened to other teams.
For tech leads
Wire evals into continuous integration, like unit tests. Every change to the system prompt, the model or the documents provided must rerun the full set and show the difference from the previous version. Version the cases alongside the code. In a startup, a script and a spreadsheet will do at first. In a large company, the cases often contain customer data, so they fall under the same access and retention rules as production.
On 220 cases per run, the interval on the difference no longer includes zero: you can tell 95% from 90%.
Go deeperWhy an average score is not enough
An overall score can rise while one category of cases falls apart. For example, a feature that goes from 88% to 91% may have lost every case where it should have said "I don't know". Group the cases by type (simple, ambiguous, off-topic, trick questions) and track each group on its own. Keep the list of failed cases too: that is what tells you where to put the effort.
Finally, be wary of small differences. On a hundred cases, a 90% score carries a margin of uncertainty of about six points either way: going from 88% to 91% may be nothing but chance. To tell 90% from 95% reliably, plan on several hundred cases: at two hundred, the gap is still borderline.
The classic mistake
fAIling RULE VI
The trap: The demo is the product
A demo that works on three examples the team picked says nothing about the cases the team never thought of. Before you show the feature to the executive committee, grow your first set to fifty real cases, test the feature on it, and present the score alongside the demo.
Read rule VI in the manifestoTry it
Take a task you already hand to an AI assistant, such as summarizing meeting notes. For a first try, gather ten real examples and write down, for each one, what a good summary must contain. Run all ten, mark each answer "correct" or "incorrect", then change a single sentence in your instructions and run them again. Compare the two scores.
Around this topic
Go deeperA model as judge
When the right result cannot be compared word for word (a tone, a summary, an explanation), you can ask a model to grade it. In October 2026, Anthropic's documentation advises a detailed rubric, a simple score ("correct" or "incorrect", or a scale from 1 to 5) and a judge that reasons before it scores. It recommends checking that the judge is reliable before using it at scale. Its examples also note that it is generally better to have the answer judged by a different model from the one that produced it. In practice, have a person grade some of the cases, and compare with the judge.
Go deeperRerunning evals on every new model
Vendors replace their models regularly. In October 2026, Anthropic gives its customers at least 60 days' notice before retiring a publicly released model, and recommends testing your applications on the replacement well before the retirement date. A new version can be better on average and worse at your task: wordier, more cautious, or different in its output format. With an eval set, the migration becomes a comparison of scores. Without one, it comes down to a few people's impressions, then to user complaints.
Go deeperBuilding test cases from production
After launch, your best test cases come from your users. An answer flagged as wrong, an abandoned conversation, a request nobody had planned for: each one can become a test case. The set then grows with the product and follows what users actually do. Production logs often contain personal data. Remove it, anonymize it or pseudonymize it before turning logs into test cases.
What to do with it
Write the success criteria before the first prototype and make them measurable: "95% of refund requests sorted correctly", not "quality answers".
Take the cases from what users really send, hard cases included, and have the person who does the task write the expected results.
Track each type of case on its own and be wary of small gaps: on a hundred cases, going from 88% to 91% may be chance.