Retrieval-augmented generation, commonly abbreviated RAG, is a technique in which a Large language model is paired with a retrieval system that fetches relevant text from an external source before the model generates its response. Rather than relying solely on knowledge encoded in its parameters during Pretraining, the model is given passages retrieved at query time, typically inserted into its Prompt, and asked to answer using that supplied material. The approach lets a general-purpose model answer questions about content it was never trained on, including private documents, recent events, or large corpora too big to fit in a Context window all at once.
The technique was introduced under the RAG name in a 2020 paper by researchers at Meta AI (then Facebook AI Research), which combined a pretrained sequence-to-sequence language model with a dense passage retriever, showing that retrieval improved factual accuracy on knowledge-intensive tasks compared to a purely parametric model. The idea built on older information-retrieval and question-answering research, but the RAG framing tied it specifically to generative transformer models and became the standard term across the industry as chat assistants and enterprise search products adopted the pattern from 2022 onward.
Architecture
A typical RAG pipeline has two stages. In the retrieval stage, a query is converted into a numerical representation, an Embedding, and compared against a precomputed index of embeddings for a document collection, most often stored in a Vector database for efficient nearest-neighbor lookup; this stage is closely related to Semantic search, which retrieves by meaning rather than exact keyword match. In the generation stage, the top-ranked passages are inserted into the model's prompt alongside the user's original question, and the model produces an answer conditioned on that retrieved context, drawing on its general language ability while being steered toward the specific supplied facts. Many production systems add further steps, such as re-ranking retrieved passages with a separate model, chunking documents into overlapping segments, or issuing multiple retrieval queries for complex questions.
Motivation: hallucination and freshness
RAG is widely used as a mitigation for Hallucination (AI), the tendency of language models to generate plausible-sounding but false or unsupported statements. Because a model's parametric knowledge is fixed at the end of pretraining and can be outdated, incomplete, or simply absent for niche and private topics, grounding a response in retrieved source text lets the model cite specific passages and reduces (though does not eliminate) the rate of fabricated claims. It also allows the same model to be applied across many private data sets, such as a company's internal documentation or a customer's own files, without any Fine-tuning, since the retrieval index can be swapped or updated independently of the model itself.
Applications and limitations
RAG underlies most enterprise "chat with your documents" products, customer support assistants grounded in a knowledge base, and coding assistants that retrieve relevant files or documentation before answering. It is also a common architectural component of AI agent systems, where an agent may issue retrieval calls as one of several available actions during a multi-step task. Limitations include sensitivity to retrieval quality, since a model can still hallucinate or answer incorrectly if the retrieved passages are irrelevant or incomplete; the added latency and infrastructure of maintaining an index; and the fact that retrieval only helps when the needed information exists somewhere in the indexed corpus. As context windows have grown into the hundreds of thousands of tokens, some practitioners have debated whether simply supplying entire documents directly in the prompt can substitute for retrieval in certain cases, though RAG remains the standard approach for corpora too large to fit in context, and for keeping large document collections searchable and current without repeated reprocessing.