AI Evals Explained: From Basics to Self-Improving Agents

Easy Mode: WTF is an eval - An eval is a test to see if what an ai agent did is right - The sauce in 'right' is encoding what you / your org thinks is right into software - Example: GTM agent that logs the customer meeting, writes follow-up draft and stops before sending - In this context an eval can be checking if the customer meeting was logged, if the follow-up draft was written, if the agent paused before sending - Evals are actually not new, they're just an evolved version of unit tests, something that's always existed in software - Recipe for setting up an eval is actually quite simple: agent + system + tasks + verifier - In the GTM agent example, you have your agent, read/write access to Salesforce, a set of tasks (like update lead status, log call, write follow-up), and a verifier (tool whose job is to determine if task was done right or wrong) - Verifiers can be a simple Python function or content/prose checks - Rubrics are a great way for codifying what good looks like for non-verifiable tasks so verifiers can still work - Once rubrics created if agent doesn't perform, root cause could be: checklist incomplete, model not smart enough, bad/incomplete context or prompt not good enough Hard Mode: WTF are Agent Environments - Environment = place an agent does work and we measure how it performs - Environments became necessary as we shifted from ChatGPT era of text in text out to agentic era of prompt in, real world work out - Example: for Claude Code/Codex, when they do work on your computer, the environment is files, browser, email, apps, permissions - When you build agent environments today = recreations simulations of existing tooling that look real to the agent but aren't real (won't overwrite prod) - Basically you want to be able to test how performant your agents are without f'ing with live systems/data in your company - Needs when building agents + evals: (1) manage tasks by team (GTM ≠ SWE), (2) team-specific environments, (3) run at scale without building all infra yourself - Popular tooling to set up agent environments is an open source framework called Harbor, which provides all of the agent environment abstractions needed to run evals so you don't manage everything, but can go low-level when needed - Harbor primitives cover: environment building, verifier building, instruction building, sandbox infra - Beginner path: grab an existing Harbor-format eval → ask Claude Code/Codex to explain → tweak for your agent - LangSmith Engine: make evals at scale; UI for people who can judge good/bad outputs and want to give feedback without living in code - Validate environments by running multiple real agents/models in them (e.g. frontier + GLM + small Qwen) and looking for weird patterns - Environment engineering is iterative like agent engineering; humans build and review outputs - To do evals at scale you need an eval suite. An eval suite is a collection of agent tasks that you're running as a company. Each task is comprised of an input prompt, a verifier, and the environment. - Anytime new model comes out, you'll run your eval suite (or a selection of tasks from the suite), and weight the tradeoffs of cost vs. speed vs. intelligence God Mode: WTF is a self-improving agent - Continuous cycle of agent in the real world → production data → turn production into evals (and use evals to improve) → continuously improve the agent - Hot take: if your team is really good at turning production data into evals, you can continuously improve agents - Evals exist to find failures; then change the agent so that failure never happens again - Two improvement buckets: harness engineering & fine tuning a small model - Harness engineering: (cheap place to start): tweak prompts, tool definitions/descriptions, model choice, model combos (e.g. Sol+Opus, GLM+Opus) - Fine-tune (on your narrow production task): open models + fine-tune infra exist; can match Opus on *that* task at ~10x cheaper; GTM agent doesn't need frontier math - God mode = collect real production data → turn into evals/environments → choose how to improve (harness first; later own fine-tuned intelligence for specific tasks) - Bottleneck often human updating verifiers/rubrics - Two questions for autonomy: how to spin faster; can it run without a human in the loop? - Goal: minimize human contact points on the loop - Agent Traces are the first practical step for orgs using agents - Trace = log of every action: tool calls - Humans can't reason well about what an agent *will* do; looking at live behavior (aka traces) makes right/wrong obvious - See a bad outcome → read traces (receipts) → find the wrong step → change agent so it doesn't repeat - Human touch points shrink with every smarter model release - Today's human role #1 (up front): explain to the mining/improvement agent what good vs bad looks like (prompt of behaviors to look for); mostly upfront with some edits over time - Then an agent can find failed tool calls, instruction-following failures, etc., and propose improvements (new prompt, new/different tool, etc.) - With a good eval suite or good prod bad-behavior measurement, an agent running overnight can be very good at proposing fixes - Human role #2: when turning production into evals/environments, review what the *company* cares about - Highest leverage organizational act: humans writing down what good looks like