# GPT-2

A large autoregressive language model released by OpenAI in stages during 2019, notable for the controversy sparked by OpenAI's initial decision to withhold its full weights over misuse concerns.

GPT-2 (Generative Pretrained Transformer 2) is a [large-language-model](https://www.wikiprompt.org/wiki/large-language-model) developed by [openai](https://www.wikiprompt.org/wiki/openai) and first announced in February 2019. Built on the [transformer](https://www.wikiprompt.org/wiki/transformer) architecture, GPT-2 was trained on WebText, a dataset of roughly 8 million web pages, using a purely [autoregressive](https://www.wikiprompt.org/wiki/autoregressive-model) next-token prediction objective, following directly on the smaller original GPT model released in 2018.

## Staged release

GPT-2 became widely known not primarily for its architecture but for how OpenAI released it. The full model contained 1.5 billion parameters, an unusually large size for a public language model at the time, and generated coherent multi-paragraph text from short prompts with a fluency that surprised many researchers. Citing concerns about malicious uses such as generating fake news, spam, and impersonation at scale, OpenAI initially declined to release the full 1.5B parameter model, publishing only smaller versions while studying potential misuse. This decision, framed publicly as the model being "too dangerous to release," drew significant criticism from parts of the research community, who argued it was either overcautious, a marketing tactic, or set an awkward precedent for withholding research results. OpenAI reversed course incrementally, releasing the full 1.5B parameter model in November 2019 after reporting that its staged-release study found no strong evidence of the misuses it had feared at that scale.

## Significance

GPT-2 was one of the first models to demonstrate strong zero-shot task performance, generating summaries, translations, and question answers without task-specific [fine-tuning](https://www.wikiprompt.org/wiki/fine-tuning), simply from being prompted with examples in natural language, an early illustration of what would later be formalized as [few-shot-learning](https://www.wikiprompt.org/wiki/few-shot-learning) and [in-context-learning](https://www.wikiprompt.org/wiki/in-context-learning) in its successor, [gpt-3](https://www.wikiprompt.org/wiki/gpt-3). Its scale, roughly ten times larger than the original GPT, also served as an early data point supporting what researchers would later describe more formally as [scaling-laws](https://www.wikiprompt.org/wiki/scaling-laws): that increasing model and data size predictably improved language modeling performance.

## Legacy

GPT-2's release strategy became a widely cited case study in AI governance and [ai-safety](https://www.wikiprompt.org/wiki/ai-safety) debate, prefiguring later, more sustained arguments over open versus closed model releases that resurfaced with systems like [llama](https://www.wikiprompt.org/wiki/llama) and other [open-weights](https://www.wikiprompt.org/wiki/open-weights) models. Within OpenAI's own product line, GPT-2's architecture and training approach formed the direct basis for [gpt-3](https://www.wikiprompt.org/wiki/gpt-3), released the following year at more than 100 times the parameter count, and ultimately for the [chatgpt](https://www.wikiprompt.org/wiki/chatgpt) product that brought the GPT lineage to mainstream attention in 2022. Researcher [alec-radford](https://www.wikiprompt.org/wiki/alec-radford) and colleagues at OpenAI are credited as GPT-2's primary authors.

---
Source: https://www.wikiprompt.org/wiki/gpt-2
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-02T20:32:39.392789+00:00