Generative Pre-trained Transformer 4 (GPT-4) is a large language model developed by OpenAI and the fourth in its series of GPT foundation models. It was released on March 14, 2023, succeeding GPT-3.5 and preceding GPT-5. GPT-4 is a multimodal model, capable of processing both text and images as input, a significant advancement over its predecessors. OpenAI has not publicly disclosed technical details about GPT-4, including the precise size of the model, its architecture, or the hardware used for training.
GPT-4 was integrated into Microsoft's Bing Chat in February 2023, before its official release in ChatGPT the following month. It was removed from ChatGPT in 2025, but remains accessible through OpenAI's API. The model's capabilities and limitations have been the subject of extensive research and public discussion.
Background
OpenAI introduced the first GPT model in 2018, based on the transformer architecture and trained on a large corpus of books. The following year, GPT-2 demonstrated the ability to generate coherent text at a larger scale. In 2020, GPT-3 was released with over 100 times as many parameters as GPT-2, and it was later refined into GPT-3.5, which powered the ChatGPT chatbot. GPT-4 built on this lineage, adding multimodal input processing and improved performance across various benchmarks.
Capabilities
The default version of GPT-4 featured an 8K context window, with a special API version supporting up to 32K tokens. Unlike earlier models, GPT-4 could accept images as input, allowing users to upload photos for analysis or ask questions about visual content. The model could also be prompted to interact with external interfaces, such as performing web searches by enclosing queries in specific tags, enabling tasks like API calls, image generation, and webpage summarization.
A 2023 article in Nature highlighted GPT-4's utility in coding assistance, despite its propensity for errors. Programmers found it helpful for identifying bugs and suggesting optimizations; one biophysicist reported reducing the time to port a program from MATLAB to Python from days to about an hour. In security tests, GPT-4 produced code vulnerable to SQL injection attacks 5% of the time, compared to 40% for GitHub Copilot in 2021.
In November 2023, OpenAI announced GPT-4 Turbo and GPT-4 Turbo with Vision, featuring a 128K context window and lower pricing.
Aptitude on standardized tests
Studies on GPT-4's performance in standardized tests have produced mixed results. In the Torrance Tests of Creative Thinking, GPT-4 scored within the top 1% for originality and fluency, with flexibility scores ranging from the 93rd to the 99th percentile.
Medical applications
Researchers from Microsoft tested GPT-4 on medical problems and found it exceeded the passing score on the USMLE by over 20 points, outperforming GPT-3.5 and models fine-tuned on medical knowledge like Med-PaLM. However, the report warned of significant risks in using large language models for medical applications, citing potential inaccuracies and hallucinations. In April 2023, Microsoft and Epic Systems announced plans to provide GPT-4-powered systems for patient inquiries and medical record analysis.
GPT-4o
On May 13, 2024, OpenAI introduced GPT-4o ("o" for "omni"), a successor that processes and generates text, audio, and image modalities in real time. GPT-4o offered rapid response times comparable to human conversation, improved performance on non-English languages, and was available to free-tier users, unlike GPT-4.
Limitations
GPT-4, like its predecessors, is known to hallucinate, producing outputs that include information not in its training data or contradicting user prompts. It also lacks transparency in decision-making; explanations of its reasoning are formed post-hoc and cannot be verified as accurate. In many cases, these explanations directly contradict previous statements.
In 2023, researchers tested GPT-4 on ConceptARC, a benchmark for abstract reasoning, and found it scored below 33% on all categories, while specialized models scored 60% on most and humans scored at least 91%. Sam Bowman, not involved in the research, noted the results may not indicate a lack of abstract reasoning, as the test is visual while GPT-4 is a language model.
Bias
GPT-4 was trained in two stages: first, on large internet text datasets to predict the next token; second, using reinforcement learning from human feedback to fine-tune behavior and refuse harmful prompts. Microsoft researchers suggested GPT-4 may exhibit cognitive biases such as confirmation bias, anchoring, and base-rate neglect.
Training
OpenAI did not release technical details of GPT-4's training. The technical report refrained from specifying model size, architecture, or hardware, citing competitive and safety implications. It described a combination of supervised learning and reinforcement learning with human and AI feedback but omitted details on dataset construction, computing power, and hyperparameters. Sam Altman stated the training cost exceeded $100 million.