Percy Liang is a computer science professor at Stanford University and director of the Center for Research on Foundation Models (CRFM) within the Stanford Institute for Human-Centered Artificial Intelligence (HAI).
Liang's early research focused on statistical and neural approaches to Natural language processing, including semantic parsing and methods for probing the robustness of NLP systems to adversarial inputs. He joined the Stanford faculty in the early 2010s and built a research group working across NLP, machine learning theory and, increasingly, the large-scale pretrained models that came to dominate the field in the years following the Transformer (architecture) architecture's introduction.
Foundation models
In 2021, Liang was a lead organizer and co-author of the Stanford paper "On the Opportunities and Risks of Foundation Models," which introduced and popularized the term Foundation model to describe large models, such as GPT-3 and BERT, trained on broad data at scale and adaptable to a wide range of downstream tasks through Fine-tuning or In-context learning. The paper argued that the shift toward foundation models represented a meaningful paradigm change in AI research and flagged risks around homogenization, since flaws or biases in a small number of widely reused foundation models could propagate across many downstream applications.
HELM and evaluation
Liang has been a prominent advocate for rigorous, transparent evaluation of language models. His group created HELM (Holistic Evaluation of Language Models), a benchmark suite designed to evaluate models across many scenarios and metrics simultaneously, including accuracy, calibration, robustness, fairness and efficiency, rather than optimizing for a single leaderboard number the way earlier benchmarks such as MMLU tended to be used. HELM was designed partly as a response to concerns about AI benchmark saturation and gaming, aiming to give a broader and more reproducible picture of model behavior across many providers' models on a common set of scenarios.
Influence
Through CRFM, Liang has continued to publish widely cited work tracking the foundation model ecosystem, including transparency indices that score major AI labs on how much they disclose about their models' training data, compute and deployment. His work sits at the intersection of empirical AI research and AI policy, aiming to bring more independent, academic scrutiny to a field increasingly dominated by results and evaluations produced by the labs building the models themselves, including OpenAI, Anthropic and Google DeepMind.