Evals Before Vibes: How We Actually Test LLM Features

The most common way teams test a language model feature is what I call vibe checking. Someone types five prompts into a playground, likes the answers, and declares it ready. Two weeks after launch, a customer finds the sixth prompt.

I am not against intuition. But intuition does not scale, and it does not survive a prompt change on a Friday afternoon. What does survive is an evaluation set, and building one is less work than most people think.

Start with fifty real examples

Not synthetic ones. Real inputs from real users, or as close as you can get. If the feature is new, pull from support tickets, sales emails, or internal documents that look like what users will send.

For each example, write down what a good output must contain. Not the perfect answer word for word, just the checks that matter. For a meeting summariser, that might be: mentions every decision, lists owners for action items, stays under 150 words.

Fifty sounds small. It is enough to catch most regressions, and small enough that one person can build it in an afternoon.

Make the checks cheap to run

Some checks can be plain code. Length limits, required fields in JSON output, forbidden phrases. Run those first because they are fast and never disagree with themselves.

def check_summary(output, case):
    errors = []
    if len(output.split()) > 150:
        errors.append('too long')
    for owner in case['owners']:
        if owner not in output:
            errors.append('missing owner ' + owner)
    return errors

For fuzzier qualities like tone or faithfulness, you can use a second model as a grader, but treat it with suspicion. I always hand-grade twenty outputs and compare them against the model grader. If the two agree less than about 85% of the time, the grader is not ready to be trusted.

Run it on every change

The whole point is repeatability. Every time someone edits the prompt, swaps the model, or changes retrieval, the eval runs and produces a score. We keep a simple table in the repo with the date, the change, and the pass rate.

On one project the table showed something nobody had noticed: a prompt tweak meant to make answers friendlier had dropped factual accuracy from 88% to 79%. Everyone liked the new tone in the playground. The numbers disagreed, and the numbers were right.

Grow the set from failures

Every bug report becomes a new test case. After three months our set had grown from 50 cases to about 220, and almost every addition came from a real complaint. That is the best kind of test suite, one written by your users.

The mindset shift

Evals are not bureaucracy. They are what let you move fast without fear. When you can change a prompt and know within ten minutes whether you made things better or worse, you experiment more, not less. Vibes feel faster, but they only feel that way until launch day.

Cover: Photo by Luke Chesser on Unsplash.