GPT-2 (Generative Pretrained Transformer 2) is a Large language model developed by OpenAI and first announced in February 2019. Built on the Transformer (architecture) architecture, GPT-2 was trained on WebText, a dataset of roughly 8 million web pages, using a purely autoregressive next-token prediction objective, following directly on the smaller original GPT model released in 2018.
Staged release
GPT-2 became widely known not primarily for its architecture but for how OpenAI released it. The full model contained 1.5 billion parameters, an unusually large size for a public language model at the time, and generated coherent multi-paragraph text from short prompts with a fluency that surprised many researchers. Citing concerns about malicious uses such as generating fake news, spam, and impersonation at scale, OpenAI initially declined to release the full 1.5B parameter model, publishing only smaller versions while studying potential misuse. This decision, framed publicly as the model being "too dangerous to release," drew significant criticism from parts of the research community, who argued it was either overcautious, a marketing tactic, or set an awkward precedent for withholding research results. OpenAI reversed course incrementally, releasing the full 1.5B parameter model in November 2019 after reporting that its staged-release study found no strong evidence of the misuses it had feared at that scale.
Significance
GPT-2 was one of the first models to demonstrate strong zero-shot task performance, generating summaries, translations, and question answers without task-specific Fine-tuning, simply from being prompted with examples in natural language, an early illustration of what would later be formalized as Few-shot learning and In-context learning in its successor, GPT-3. Its scale, roughly ten times larger than the original GPT, also served as an early data point supporting what researchers would later describe more formally as Scaling laws: that increasing model and data size predictably improved language modeling performance.
Legacy
GPT-2's release strategy became a widely cited case study in AI governance and AI safety debate, prefiguring later, more sustained arguments over open versus closed model releases that resurfaced with systems like Llama and other Open-weights models models. Within OpenAI's own product line, GPT-2's architecture and training approach formed the direct basis for GPT-3, released the following year at more than 100 times the parameter count, and ultimately for the ChatGPT product that brought the GPT lineage to mainstream attention in 2022. Researcher Alec Radford and colleagues at OpenAI are credited as GPT-2's primary authors.