Hallucination, in the context of artificial intelligence, refers to the tendency of a generative model, especially a large language model, to produce statements that are confident and fluent but factually incorrect, fabricated, or unsupported by its training data or any source it was given. The term is borrowed loosely from human psychology and is a subject of some debate among researchers, since it implies a perceptual error rather than the more mechanical process by which a language model generates text.
Causes
Language models are trained to predict plausible continuations of text, not to verify facts, so a model has no built-in mechanism to distinguish a well-supported claim from a fluent but invented one. Hallucination is more likely when a prompt asks about information that appeared rarely or inconsistently in Training data, when a question falls outside a model's Context window or knowledge cutoff, or when a model is asked to cite specific sources, dates, or numbers it was never trained to memorize precisely, such as legal case citations or academic references. Some researchers attribute part of the problem to RLHF, arguing that training a model to sound confident and satisfy human raters can reward fluent guessing over admitting uncertainty. Hallucination is closely related to the broader Grounding (AI) problem: a model trained purely on text has no direct connection between its outputs and the state of the world, so nothing in its training process guarantees factual accuracy.
Notable incidents
Hallucination moved from an academic concern to a public one after ChatGPT's public launch in late 2022 brought large language models to a mass audience. In a widely reported 2023 case, lawyers in a New York federal lawsuit submitted a brief containing fabricated case citations that ChatGPT had invented, leading to sanctions and drawing wide attention to the risk of trusting model outputs without verification. Search engines and assistants that summarize web results have also produced hallucinated summaries, prompting companies to add citations and confidence caveats to AI-generated answers.
Mitigation
The most widely deployed mitigation is Retrieval-augmented generation, in which a system retrieves relevant documents at query time and instructs the model to base its answer on them, reducing but not eliminating the rate of fabricated claims. Other approaches include Fine-tuning a model specifically to express calibrated uncertainty, prompting techniques that ask a model to cite sources or show its reasoning via Chain-of-thought, and post-hoc fact-checking pipelines that verify claims against external databases before showing them to a user. Benchmarks and evaluation suites increasingly include hallucination rate as a tracked metric alongside general capability, and some labs report hallucination statistics for new model releases as part of their evaluation process.
Ongoing debate
Researchers disagree about whether hallucination is a solvable engineering problem or an inherent property of how autoregressive language models generate text. Some argue that scaling and better training data will steadily reduce hallucination rates, pointing to measured improvements across successive model generations. Others argue that because a language model has no ground-truth verification step built into how it generates each token, some rate of fabrication is unavoidable without external tools, and that the practical goal should be building interfaces and workflows that assume hallucination will happen and are designed to catch it.