# Foundation model

A foundation model is a large machine learning model trained on broad data at scale that can be adapted, through fine-tuning or prompting, to a wide range of downstream tasks, a term coined by Stanford researchers in 2021.

A foundation model is a machine learning model trained on broad, often web-scale, data that can be adapted to a wide range of downstream tasks rather than being built for a single narrow purpose. The term was coined in a 2021 report by Stanford's Center for Research on Foundation Models (CRFM), led by researchers including [Percy Liang](https://www.wikiprompt.org/wiki/percy-liang), who argued that models such as [GPT-3](https://www.wikiprompt.org/wiki/gpt-3) and [BERT](https://www.wikiprompt.org/wiki/bert) represented a new paradigm: a single model, once pretrained, serves as the foundation for many applications through [fine-tuning](https://www.wikiprompt.org/wiki/fine-tuning), [in-context learning](https://www.wikiprompt.org/wiki/in-context-learning), or [prompting](https://www.wikiprompt.org/wiki/prompt-engineering), rather than each task requiring a model trained from scratch.

## Origins and definition

The CRFM report distinguished foundation models by their combination of scale, in parameters, data, and compute, broad training objectives such as [self-supervised](https://www.wikiprompt.org/wiki/self-supervised-learning) [pretraining](https://www.wikiprompt.org/wiki/pretraining) on unlabeled data, and generality, meaning a single model's learned representations transfer across many downstream tasks with relatively light adaptation. This framing built on and extended the earlier idea of [transfer-learning](https://www.wikiprompt.org/wiki/transfer-learning), where a model pretrained on one task or dataset is reused, typically after fine-tuning, on a related task.

## Examples

Foundation models span modalities: text-focused models like the [GPT](https://www.wikiprompt.org/wiki/gpt-4) series, [Claude](https://www.wikiprompt.org/wiki/claude), [Gemini](https://www.wikiprompt.org/wiki/gemini), and [Llama](https://www.wikiprompt.org/wiki/llama); vision-language systems built on [CLIP](https://www.wikiprompt.org/wiki/clip)-style pretraining; image generation models such as [Stable Diffusion](https://www.wikiprompt.org/wiki/stable-diffusion); and speech models like [Whisper](https://www.wikiprompt.org/wiki/whisper). Most [large language models](https://www.wikiprompt.org/wiki/large-language-model) are foundation models in this sense, though the term predates and is broader than LLM, since it also covers non-text and [multimodal](https://www.wikiprompt.org/wiki/multimodal-ai) systems.

## Why the term is used

The label emerged partly to capture a shift in how AI systems are built and deployed: instead of a research group training a bespoke model per task, an [open-weights](https://www.wikiprompt.org/wiki/open-weights) or proprietary foundation model is released once and then adapted by many downstream users, dramatically lowering the cost of building new AI applications. This gave rise to an ecosystem, exemplified by platforms like [Hugging Face](https://www.wikiprompt.org/wiki/hugging-face), centered on sharing, fine-tuning, and deploying foundation models rather than training from scratch. The term also proved useful in policy discussions, since regulations such as the [EU AI Act](https://www.wikiprompt.org/wiki/eu-ai-act) created specific obligations for providers of general-purpose or foundation models, distinct from developers who merely build applications on top of them.

## Criticism and caveats

The CRFM report itself acknowledged risks: foundation models concentrate capability and influence in the hands of the few organizations able to train them, propagate biases present in their broad training data across every downstream application built on them, and can behave unpredictably outside the distribution of their training data. Critics, including some who otherwise welcomed the framing, noted that "foundation" implies a stability and reliability that early large models did not always have, and that the term can obscure meaningful differences between models, such as whether they are open or closed, or trained with different [alignment](https://www.wikiprompt.org/wiki/alignment) techniques. Despite this, foundation model became standard vocabulary in both research papers and regulation within a few years of its introduction.

## Relationship to other terms

Foundation model overlaps heavily with, but is not identical to, [frontier model](https://www.wikiprompt.org/wiki/frontier-model), which specifically denotes the most capable models at a given time, and with [LLM](https://www.wikiprompt.org/wiki/large-language-model), which is specific to text. A foundation model need not be at the frontier of capability, and not all foundation models are language models.

---
Source: https://www.wikiprompt.org/wiki/foundation-model
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-02T20:30:04.686695+00:00