How Do You Know Your AI Feature Is Any Good? Evals with Rubrics

The AI feature is built. The demo went great, the client was thrilled, everything is live. Then two weeks later you tweak a small thing in the prompt – and wonder: did the output just get better? Worse? How would I even know?

This is exactly where classic testing stops working. A unit test checks: input X produces exactly output Y. But an LLM phrases things differently every single time. All of them are somehow right, none of them are identical. assertEquals has nothing to grab onto.

What most teams do instead: they read a few outputs, nod, and ship. That's not testing. That's hoping.

The Problem: "Looks Good" Doesn't Scale

At first it works. You read ten generated answers, they seem reasonable, done. But this approach collapses in three places:

After every prompt change you'd have to re-read everything. You won't, of course. So regressions creep in that nobody notices – until a user complains.
You read with rose-tinted glasses. People see what they want to see, especially with their own prompt. A systematic flaw hiding in every fifth output doesn't show up while scrolling.
Model updates happen without you. The provider swaps the model under the hood, your prompt stays the same, your output doesn't. Without measurement, you simply won't notice.

It happened to me: a prompt change meant to improve one case quietly degraded three others. I only noticed days later. Since then I treat LLM output like any other code: it gets tested. Just differently.

Rubrics: A Scoring Grid Instead of Gut Feeling

The solution is unspectacular and comes from school: a rubric – a structured scoring grid. Instead of asking "is this output good?", you break "good" down into individual, checkable criteria.

Example: a feature that generates reply suggestions for customer inquiries. Its rubric:

Answers the actual question – not a similar one, the actual one.
Invents nothing – no prices, deadlines, or features that don't exist.
Hits the tone – professional, but not stiff.
Keeps the length – three to five sentences, not an essay.
Escalates when in doubt – legal questions or complaints get referred to a human.

Each criterion is judged individually as pass or fail. No point scales – more on that in a moment. The key shift: "good" is no longer a matter of taste, it's a checklist. You can argue about the criteria – and that's exactly the point. The argument happens once, while writing the rubric, not all over again for every single output.

LLM-as-Judge: The Model Grades Along

Now the part that sounds circular at first: the grading is done by another LLM. You hand a second model the output plus the rubric and have it check each criterion.

Why does this work? Because judging is easier than generating. A model that invents a price while generating will quite reliably notice that someone else's text contains a price that doesn't appear in the source data. Verifying is the easier task – the same way proofreading is easier than writing.

That turns gut feeling into a test run: 30 typical inputs, each pushed through the feature, every output checked by the judge against the rubric. After every prompt change, the set runs again. 28 out of 30 passed before, 23 after? Now you know – before a user does.

What I Learned Along the Way

Pass/fail beats point scales. "Rate the tone from 1 to 10" gives you a 7 today and an 8 tomorrow for the same text. "Is the tone professional without being stiff – yes or no?" stays stable. Scales drift, decisions don't.
One criterion per question. A judge asked to check five things at once will miss some. Five separate checks are more reliable than one big one – and you can see which criterion is slipping.
Starting small is enough. 20 to 50 test cases cover a surprising amount. What matters more than volume: real cases from production, especially the uncomfortable ones. Every time something goes wrong in the wild, that case goes into the set.
The rubric is code. It lives in the repo, it's versioned, changes go through review. A rubric gathering dust in a Google Doc isn't one.
The judge doesn't replace the human. It scales the human. I still read spot checks – but the judge reads everything. And every now and then I audit the judge itself: does it grade the cases the way I would? If not, the rubric gets sharpened – the verdict doesn't get ignored.

When It's Worth the Effort

Honest answer: not always. For an internal prototype whose output a human reviews anyway, an eval pipeline is overkill.

For me the threshold is the same as with tests: the moment a prompt change gives you that uneasy feeling because you don't know what it might break – it's time. In software, we call that regression testing. For LLM features, it's called evals. It's the same need: being able to change things without being afraid.

And just like with tests: skipping them doesn't save the time. It just moves it – from development to debugging, after the user has complained.

Conclusion: Measure, Don't Hope

LLM features feel like magic at first, and you don't test magic – you marvel at it. That's exactly the trap. The output isn't magical, it's just non-deterministic. And non-determinism is not a reason to stop measuring. It's the reason to measure differently.

A rubric with clear criteria, a judge that checks them, a test set built from real cases – that's all it takes to start. No framework, no platform, no data science department.

An AI feature without evals is a car without a speedometer. It drives – you just don't know how fast, and you only find out it was too fast when you hear the crash.