GPT-4 is a large language model developed by OpenAI and the fourth in its series of GPT foundation models. Released on March 14, 2023, it succeeded GPT-3.5 and was followed by its successor GPT-5. Unlike its predecessors, GPT-4 is a multimodal model capable of processing both text and image inputs. OpenAI has not revealed technical details about the model, including its precise size.
Background
OpenAI introduced the first GPT model (GPT-1) in 2018, publishing a paper titled "Improving Language Understanding by Generative Pre-Training," which was based on the transformer architecture and trained on a large corpus of books. In 2019, they introduced GPT-2, a larger model capable of generating coherent text. In 2020, they released GPT-3, a model with over 100 times as many parameters as GPT-2. GPT-3 was further refined into GPT-3.5, which was used to create the ChatGPT Chatbot.
Capabilities
GPT-4 was released on March 14, 2023. The default version featured an 8K context window, with a special version in the API supporting up to 32K tokens. Unlike its predecessors, GPT-4 is a multimodal model: it can take images as well as text as input. It can interact with users through spoken words and respond to images, allowing it to provide suggestions or answers based on photo uploads.
GPT-4 can be prompted to interact with external interfaces. For example, the model could be told to enclose a query within <search></search> tags to perform a web search, the result of which would be inserted into the model's prompt to allow it to form a response. This enables the model to perform tasks beyond its normal text-prediction capabilities, such as using APIs, generating images, and accessing and summarizing webpages.
A 2023 article in Nature stated programmers have found GPT-4 useful for assisting in coding tasks (despite its propensity for error), such as finding errors in existing code and suggesting optimizations to improve performance. The article quoted a biophysicist who found that the time he required to port one of his programs from MATLAB to Python went down from days to "an hour or so". On a test of 89 security scenarios, GPT-4 produced code vulnerable to SQL injection attacks 5% of the time, an improvement over GitHub Copilot from 2021, which produced vulnerabilities 40% of the time.
In November 2023, OpenAI announced the GPT-4 Turbo and GPT-4 Turbo with Vision model, which featured a 128K context window and significantly cheaper pricing.
Aptitude on standardized tests
Studies have disputed the reliability of running chatbots through standardized tests. In the Torrance Tests of Creative Thinking, GPT-4 scored within the top 1% for originality and fluency, while its flexibility scores ranged from the 93rd to the 99th percentile.
Medical applications
Researchers from Microsoft tested GPT-4 on medical problems and found that "GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge (Med-PaLM, a prompt-tuned version of Flan-PaLM 540B)." Despite GPT-4's strong performance on tests, the report warned of "significant risks" of using large language models in medical applications, as they may provide inaccurate recommendations and hallucinate major factual errors.
In April 2023, Microsoft and Epic Systems announced they would provide healthcare providers with GPT-4-powered systems for assisting in responding to questions from patients and analyzing medical records.
GPT-4o
On May 13, 2024, OpenAI introduced GPT-4o ("o" for "omni"), a successor to GPT-4 that marked a significant advancement by processing and generating outputs across text, audio, and image modalities in real time. GPT-4o exhibited rapid response times comparable to human reaction in conversations, substantially improved performance on non-English languages, and enhanced understanding of vision and audio. It was also available to free-tier users, unlike GPT-4.
Limitations
Like its predecessors, GPT-4 has been known to hallucinate, meaning its outputs may include information not in the training data or that contradicts the user's prompt.
GPT-4 also lacks transparency in its decision-making processes. If requested, the model can provide an explanation as to how and why it makes its decisions, but these explanations are formed post-hoc; it is impossible to verify if those explanations truly reflect the actual process. In many cases, when asked to explain its logic, GPT-4 gives explanations that directly contradict its previous statements.
In 2023, researchers tested GPT-4 against a new benchmark called ConceptARC, designed to measure abstract reasoning, and found it scored below 33% on all categories, while models specialized for similar tasks scored 60% on most, and humans scored at least 91% on all. Sam Bowman, who was not involved in the research, said the results do not necessarily indicate a lack of abstract reasoning abilities because the test is visual, while GPT-4 is a language model.
Bias
GPT-4 was trained in two stages. First, the model was given large datasets of text taken from the internet and trained to predict the next token (roughly corresponding to a word) in those datasets. Second, human reviews were used to fine-tune the system in a process called reinforcement learning from human feedback, which trains the model to refuse prompts that go against OpenAI's definition of harmful behavior, such as questions on how to perform illegal activities, advice on how to harm oneself or others, or requests for descriptions of graphic, violent, or sexual content.
Microsoft researchers suggested GPT-4 may exhibit cognitive biases such as confirmation bias, anchoring, and base-rate neglect.
Training
OpenAI did not release the technical details of GPT-4; the technical report explicitly refrained from specifying the model size, architecture, or hardware used during either training or inference. While the report described that the model was trained using a combination of supervised learning on a large dataset, then reinforcement learning using both human and AI feedback, it did not provide details of the training, including the process by which the training dataset was constructed, the computing power required, or any hyperparameters such as the learning rate, epoch count, or optimizers used. The report claimed that "the competitive landscape and the safety implications of large-scale models" were factors that influenced this decision.
Sam Altman stated that the cost of training GPT-4 was more than $100 million.
Reception and Impact
GPT-4 was integrated by Microsoft into Bing Chat, which launched in February 2023, before the model's official release. The model quickly became a foundation for numerous applications and services, demonstrating the potential of GPT models across various domains.
Availability
GPT-4 was released in ChatGPT in March 2023 and remained available until it was removed in 2025. The model was removed from the OpenAI API on October 23, 2026.