Chain-of-thought is a prompting technique in which a language model is asked, or encouraged, to produce a sequence of intermediate reasoning steps before giving a final answer, rather than jumping directly to a conclusion. The technique improves accuracy on tasks that require multiple steps of arithmetic, logic, or commonsense reasoning, and became one of the most influential ideas in Prompt engineering after it was formally described in 2022.
Origins
The term was introduced in a January 2022 paper, "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," by Jason Wei and coauthors at Google. The paper showed that including a small number of worked examples in a prompt, each with the reasoning steps written out rather than only the final answer, dramatically improved a model's performance on math word problems and other multi-step tasks, a form of Few-shot learning applied through the prompt itself. Crucially, the paper reported that this benefit only appeared in sufficiently large models, tying chain-of-thought closely to the discussion of Emergent abilities and Scaling laws.
Zero-shot chain-of-thought
Later in 2022, a separate paper by Takeshi Kojima and coauthors found that a much simpler prompt, appending the phrase "let's think step by step" to a question with no worked examples at all, produced a similar improvement in reasoning accuracy on many tasks. This result, known as zero-shot chain-of-thought, showed that the benefit did not strictly require few-shot examples; simply instructing the model to reason before answering was often enough to unlock better performance, an instance of zero-shot behavior building on capabilities the model already had from pretraining.
Mechanism and self-consistency
Researchers have offered several explanations for why chain-of-thought helps, including that it gives the model more computational steps to work with before committing to an answer, that it lets the model use its own earlier reasoning as additional context via In-context learning, and that it reduces the chance of an error compounding silently inside a single forward pass. A related technique called self-consistency generates multiple independent chains of reasoning for the same question and takes the most common final answer among them, further improving accuracy at the cost of additional computation.
From prompting technique to trained behavior
Chain-of-thought began as a prompting strategy applied to models that were not specifically trained to reason step by step, but it became foundational to a new class of systems. Starting with OpenAI o1 in 2024, labs began training models with reinforcement learning specifically to produce long, effective chains of reasoning before answering, rather than relying on a user to request that behavior through prompting. This shift gave rise to the category of reasoning models, including DeepSeek-R1, and reframed chain-of-thought from an optional prompting trick into a trained, often hidden, internal process, with the amount of reasoning a model performs becoming a new lever, called Test-time compute, for improving accuracy at the cost of additional inference time.