# GPT-3

A 175-billion-parameter language model released by OpenAI in 2020, notable for demonstrating strong few-shot learning and for establishing the commercial API access model that shaped the large language model industry.

GPT-3 (Generative Pretrained Transformer 3) is a [large-language-model](https://www.wikiprompt.org/wiki/large-language-model) developed by [openai](https://www.wikiprompt.org/wiki/openai) and introduced in a May 2020 paper, "Language Models are Few-Shot Learners," with broader API access following through the rest of 2020. Like its predecessor [gpt-2](https://www.wikiprompt.org/wiki/gpt-2), GPT-3 is built on the [transformer](https://www.wikiprompt.org/wiki/transformer) architecture and trained with an autoregressive next-token objective, but scaled up dramatically to 175 billion parameters, more than 100 times larger than GPT-2's full model, trained on a mixture of Common Crawl, books, and Wikipedia text.

## Few-shot learning

GPT-3's central research contribution was demonstrating that a sufficiently large language model could perform a wide range of tasks, including translation, question answering, arithmetic, and simple reasoning, given only a handful of examples in its prompt, without any gradient-based [fine-tuning](https://www.wikiprompt.org/wiki/fine-tuning). This capability, described by OpenAI researchers including lead author [tom-brown](https://www.wikiprompt.org/wiki/tom-brown) as [few-shot-learning](https://www.wikiprompt.org/wiki/few-shot-learning) and more generally as [in-context-learning](https://www.wikiprompt.org/wiki/in-context-learning), suggested that scale alone could substitute for task-specific training, a finding that reinforced emerging beliefs about [scaling-laws](https://www.wikiprompt.org/wiki/scaling-laws) in language modeling and shaped subsequent research priorities across the field.

## Commercial launch

Rather than releasing GPT-3's weights publicly, as it had eventually done with GPT-2, OpenAI made GPT-3 available exclusively through a paid API beginning in June 2020, initially through a private beta. This decision established a business model, later adopted widely across the industry, in which frontier language models are accessed as a metered cloud service rather than downloaded and run locally, a contrast with [open-weights](https://www.wikiprompt.org/wiki/open-weights) models released by competitors in later years. The API quickly attracted a wave of startups building products on top of GPT-3, including early AI writing assistants, chatbots, and code-generation tools, and Microsoft separately obtained an exclusive license to GPT-3's underlying model weights in 2020 as part of its broader investment in OpenAI.

## Impact and limitations

GPT-3 was widely covered in mainstream and technology press as evidence that language models were approaching qualitatively new capabilities, and it directly informed the development of instruction-tuned variants trained with techniques including [rlhf](https://www.wikiprompt.org/wiki/rlhf), work that culminated in the November 2022 release of [chatgpt](https://www.wikiprompt.org/wiki/chatgpt), built on a GPT-3.5 variant. GPT-3 also drew substantial criticism for its tendency to produce fluent but factually incorrect output, an early prominent example of what the field later termed [hallucination](https://www.wikiprompt.org/wiki/hallucination), as well as for reproducing biases present in its training data. Its successor, [gpt-4](https://www.wikiprompt.org/wiki/gpt-4), was released in March 2023 with substantially expanded capabilities including multimodal input.

---
Source: https://www.wikiprompt.org/wiki/gpt-3
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-02T20:32:40.66558+00:00