# OpenAI GPT-3 Launch

GPT-3 is a large language model released by OpenAI in June 2020, with 175 billion parameters, noted for its zero-shot and few-shot abilities. Its API launch enabled broad access, and Microsoft later licensed the model exclusively.

Generative Pre-trained Transformer 3 (GPT-3) is a large language model released by OpenAI in 2020 as part of the company's GPT series of models. It is a decoder-only [transformer](https://www.wikiprompt.org/wiki/transformer) model of a [deep neural network](https://www.wikiprompt.org/wiki/deep-learning), using an attention mechanism to focus selectively on relevant input segments. With 175 billion parameters, each stored in 16-bit precision, GPT-3 requires 350GB of storage and supports a context window of 2,048 tokens. It demonstrated strong zero-shot and few-shot learning across many tasks, and its release via an API was announced on June 11, 2020.

## Background

According to The Economist, improved algorithms, more powerful computers, and a rising digitized corpus fueled a revolution in [machine-learning](https://www.wikiprompt.org/wiki/machine-learning). New techniques in the 2010s led to rapid improvements in language manipulation, relying on neural architectures loosely based on the brain. The [transformer](https://www.wikiprompt.org/wiki/transformer) architecture, introduced in 2017, became foundational for [natural-language-processing](https://www.wikiprompt.org/wiki/natural-language-processing) systems. In June 2018, OpenAI published the first generative pre-trained transformer (GPT-1), a model trained on a large text corpus and then fine-tuned for specific tasks. GPT-2 followed in February 2019, scaling parameters and dataset size tenfold, reaching 1.5 billion parameters trained on 8 million web pages. In February 2020, Microsoft introduced Turing NLG, a 17-billion-parameter model, then considered the largest.

## Training and capabilities

On May 28, 2020, an arXiv preprint by 31 OpenAI engineers and researchers described GPT-3's development, increasing model capacity by over two orders of magnitude from GPT-2, making it the largest non-sparse language model of that period. Its accuracy stemmed from increased capacity and parameters, being ten times larger than Microsoft's Turing NLG. Lambdalabs estimated training cost at $4.6 million and 355 years on a single GPU in 2020. Pre-training data included 410 billion byte-pair encoded tokens from filtered Common Crawl (60% of weighted data), plus WebText2, Books1, Books2, and Wikipedia (3%). GPT-3 also could code in Python, JSX, and CSS.

GPT-3 lacked task-specific fine-tuning, instead handling broad instructions. However, training data contained toxic language that the model could sometimes reproduce. A University of Washington study found GPT-3's toxicity comparable to GPT-2 and CTRL. OpenAI implemented mitigation strategies, reducing toxic outputs compared to GPT-1.

## API access and evaluation

On June 11, 2020, OpenAI released a general-purpose API with a text-in text-out interface, enabling any English language task, rather than single-use cases. Early testers described GPT-3 as eerily good at coherent text generation. In an experiment with 80 US participants, only a 52% correct rate distinguished GPT-3 articles from human-authored ones, a slight edge over random guessing. By November 18, 2021, OpenAI broadened access to its API with a content moderation tool. On January 27, 2022, InstructGPT became the default API model, improving instruction following and reducing fabricated facts.

## Licensing and impact

On September 22, 2020, Microsoft announced an exclusive license for GPT-3's underlying model. Others could use the public API but only Microsoft could access the code and weights. GPT-3 demonstrated potential for generating news articles hard to distinguish from human writing, prompting concerns about misinformation and fraud. The researchers described potential harmful effects including spam and legal abuse.

## See also
* [openai](https://www.wikiprompt.org/wiki/openai)
* [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)


---
Source: https://www.wikiprompt.org/wiki/openai-gpt-3-launch
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-12T16:21:42.093797+00:00