LLM evaluation, commonly called evals, refers to the methods and infrastructure used to measure the capability, quality, reliability, and safety of large language models. As LLMs have been applied to an increasingly broad range of tasks, evaluation has grown from a narrow academic exercise into a major practical discipline within AI labs, since evals determine what gets reported publicly, what guides model selection by developers, and increasingly what feeds back into training itself.
Static benchmark suites
The most traditional form of evaluation runs a model against a fixed benchmark dataset with known correct answers, such as MMLU for knowledge, HumanEval or SWE-bench for coding, and ARC-AGI for abstract reasoning. These suites are cheap to run repeatedly and allow direct numerical comparison, but suffer from benchmark saturation and training data contamination as models are trained on increasingly large web scrapes that may overlap with test sets.
Human preference arenas
Because many valuable LLM qualities, such as helpfulness, writing quality, or tone, resist simple correctness scoring, platforms like LMArena, formerly Chatbot Arena, collect large volumes of blind, head-to-head human comparisons between model outputs and compute an Elo-style ranking. This approach captures aggregate human preference well but has been criticized for rewarding stylistic traits, such as verbosity or agreeableness, that do not necessarily track deeper capability or correctness, and for being possible to game by tuning a model specifically toward the platform's user distribution.
LLM-as-judge
A newer approach uses one LLM to evaluate another's outputs, scoring responses for correctness, helpfulness, or adherence to a rubric, since human evaluation is slow and expensive at scale. LLM-as-judge methods can approximate human judgment reasonably well on many tasks and allow evaluation to scale to far more examples than human raters could cover, but inherit their own model's biases and blind spots, including a tendency to favor longer or more confidently worded answers, and can be exploited if a model being evaluated learns to produce outputs that game the judge rather than genuinely improve.
Domain-specific and safety evals
Beyond general capability, evaluation increasingly covers narrower properties: hallucination rates on factual questions, susceptibility to jailbreaks and prompt injection, bias across demographic groups, and dangerous-capability evaluations relevant to AI safety, such as whether a model can meaningfully assist with cyberattacks or bioweapons synthesis. Labs including Anthropic and OpenAI run such evaluations as part of pre-deployment testing tied to their responsible scaling commitments, and third-party bodies, including government AI safety institutes formed after the Bletchley Declaration, conduct independent evaluations of frontier releases.
Challenges
Evaluation faces persistent difficulties: benchmarks and arenas can be gamed once a lab knows what is being measured, a version of Goodhart's law applied to model development; contamination undermines the validity of static scores; and no single number captures the many, sometimes conflicting, qualities users care about, such as helpfulness, honesty, and harmlessness. This has pushed the field toward evaluation portfolios combining multiple methods, ongoing "living benchmarks" that refresh their questions, and greater emphasis on real-world, task-based, and agentic evaluations that better reflect deployed use than isolated question-answering.
Significance
Because evals shape which models are perceived as best, and increasingly feed directly into training through techniques like RLHF and reward modeling, the design of evaluation methodology has become a significant, if less visible, driver of how the field's leading models actually behave.