A foundation model is a machine learning model trained on broad, often web-scale, data that can be adapted to a wide range of downstream tasks rather than being built for a single narrow purpose. The term was coined in a 2021 report by Stanford's Center for Research on Foundation Models (CRFM), led by researchers including Percy Liang, who argued that models such as GPT-3 and BERT represented a new paradigm: a single model, once pretrained, serves as the foundation for many applications through Fine-tuning, in-context learning, or prompting, rather than each task requiring a model trained from scratch.
Origins and definition
The CRFM report distinguished foundation models by their combination of scale, in parameters, data, and compute, broad training objectives such as self-supervised pretraining on unlabeled data, and generality, meaning a single model's learned representations transfer across many downstream tasks with relatively light adaptation. This framing built on and extended the earlier idea of Transfer learning, where a model pretrained on one task or dataset is reused, typically after fine-tuning, on a related task.
Examples
Foundation models span modalities: text-focused models like the GPT series, Claude, Gemini, and Llama; vision-language systems built on CLIP-style pretraining; image generation models such as Stable Diffusion; and speech models like Whisper. Most large language models are foundation models in this sense, though the term predates and is broader than LLM, since it also covers non-text and multimodal systems.
Why the term is used
The label emerged partly to capture a shift in how AI systems are built and deployed: instead of a research group training a bespoke model per task, an open-weights or proprietary foundation model is released once and then adapted by many downstream users, dramatically lowering the cost of building new AI applications. This gave rise to an ecosystem, exemplified by platforms like Hugging Face, centered on sharing, fine-tuning, and deploying foundation models rather than training from scratch. The term also proved useful in policy discussions, since regulations such as the EU AI Act created specific obligations for providers of general-purpose or foundation models, distinct from developers who merely build applications on top of them.
Criticism and caveats
The CRFM report itself acknowledged risks: foundation models concentrate capability and influence in the hands of the few organizations able to train them, propagate biases present in their broad training data across every downstream application built on them, and can behave unpredictably outside the distribution of their training data. Critics, including some who otherwise welcomed the framing, noted that "foundation" implies a stability and reliability that early large models did not always have, and that the term can obscure meaningful differences between models, such as whether they are open or closed, or trained with different alignment techniques. Despite this, foundation model became standard vocabulary in both research papers and regulation within a few years of its introduction.
Relationship to other terms
Foundation model overlaps heavily with, but is not identical to, frontier model, which specifically denotes the most capable models at a given time, and with LLM, which is specific to text. A foundation model need not be at the frontier of capability, and not all foundation models are language models.