A large language model (LLM) is a Neural network, typically built on the Transformer (architecture) architecture, trained on very large quantities of text to predict the next unit of text, or token, in a sequence. Despite this narrow training objective, LLMs trained at sufficient scale exhibit broad abilities in language understanding, generation, translation, summarization, and reasoning, and form the technical basis of assistants such as ChatGPT, Claude (AI model family), and Gemini.
History
Transformer-based language models emerged shortly after the architecture's 2017 introduction. BERT (2018) demonstrated the value of large-scale pretraining for language understanding tasks, while OpenAI's GPT series pursued generative, autoregressive training at increasing scale, culminating in GPT-3 (2020), whose 175 billion parameters and strong few-shot learning performance drew wide attention. GPT-4 (2023) extended the paradigm with multimodal input and far greater capability, and the November 2022 release of ChatGPT, built on the GPT-3.5 model, brought conversational LLMs to mainstream use, prompting rapid entry by competitors including Google's Gemini family and Meta's open-weights Llama series.
How they work
LLMs are trained in a Pretraining phase on vast, largely uncurated text corpora using a self-supervised objective, learning to predict the next token given prior context. This produces an autoregressive model capable of generating fluent text one token at a time. Pretrained "base" models are typically adapted further through Fine-tuning and reinforcement learning from human feedback to follow instructions and converse safely, a process central to models like ChatGPT and Claude. The amount of text a model can process in one interaction is bounded by its Context window, which has grown from roughly a few thousand tokens in early models to over a million in some 2025-era systems.
Scale and capability
Empirical relationships known as Scaling laws describe how a model's loss improves predictably as its parameter count, training data, and compute increase together. Some capabilities, such as multi-step arithmetic or following complex instructions, appear to emerge relatively abruptly past certain scale thresholds, a phenomenon called Emergent abilities, though researchers dispute how much of this is a real property of the models versus an artifact of how capabilities are measured.
Limitations
LLMs are prone to Hallucination (AI), producing fluent but factually incorrect statements with no signal to the user that the output is unreliable. Because they generate text based on learned statistical patterns rather than verified knowledge, they can also reproduce biases present in training data and remain vulnerable to adversarial prompting techniques such as jailbreaks. Mitigations include grounding outputs in retrieved external documents, a technique known as retrieval-augmented generation, and increasingly, "reasoning models" that use additional test-time computation to check their own work before answering.