Dan Hendrycks is an American computer scientist and the executive and research director of the Center for AI Safety (CAIS), a nonprofit organization focused on reducing large-scale risks from artificial intelligence. He earned a PhD in computer science at the University of California, Berkeley, where his research on robustness and machine learning evaluation produced some of the most widely used benchmarks in the field.
Benchmarks
While a graduate student, Hendrycks led the creation of the Massive Multitask Language Understanding benchmark, known as MMLU, first released in 2020. MMLU tests models across 57 academic and professional subjects and became one of the standard yardsticks for LLM evaluation, cited in nearly every major Large language model release since GPT-3. He also contributed to other widely used evaluation datasets, including MATH, a benchmark of competition mathematics problems, and helped popularize methodology for measuring Hallucination (AI) and robustness in Machine learning systems more broadly.
Center for AI Safety and the extinction statement
Hendrycks founded CAIS to fund technical safety research and to raise the profile of catastrophic AI risk as a mainstream policy concern, publishing academic and policy work on topics including Reward hacking, model robustness, and AI alignment. In May 2023, CAIS published the "Statement on AI Risk," a single sentence stating that mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. The statement was signed by hundreds of researchers and executives, including leaders of OpenAI, Anthropic, and Google DeepMind, and was widely covered as a sign that concern about Existential risk from AI had moved from a fringe position to one held across the mainstream AI research community.
Hendrycks later became a safety adviser to xAI, Elon Musk's AI company, drawing some criticism from commentators who questioned the compatibility of that role with his public risk advocacy. He also co-authored an open textbook, "Introduction to AI Safety, Ethics, and Society," intended to formalize AI safety as an academic subject taught alongside traditional machine learning coursework.
Reception
Hendrycks is generally regarded within the field as occupying a middle position between the more categorical predictions of researchers such as Eliezer Yudkowsky and industry figures skeptical of near-term catastrophic risk. His empirical contributions, particularly MMLU, are cited independently of his advocacy work and are used by researchers across the spectrum of views on AI risk, including labs that publicly disagree with parts of his policy positions. Critics have questioned whether benchmarks like MMLU are becoming saturated or gamed as models are increasingly trained on data overlapping with evaluation sets, a critique Hendrycks has acknowledged and addressed with follow-up, harder benchmarks.