Test-time compute refers to the amount of computation a model performs while producing an answer, as distinct from the computation spent during training. For most of the deep learning era, capability gains came almost entirely from scaling up training: more parameters, more data, more training compute, as described by Scaling laws. Test-time compute treats inference itself as a second lever. A model can be made to think longer, explore more candidate answers, or check its own work before returning a response, and each of those costs additional compute at the moment of use rather than at training time.
The approach entered mainstream use with OpenAI o1, released by OpenAI in September 2024. Unlike earlier chatbots tuned to answer quickly in a single pass, o1 was trained to generate an extended, largely hidden chain of internal reasoning before producing a final answer, and its accuracy on math, coding, and logic benchmarks improved measurably as more reasoning tokens were allowed. This established the Reasoning model as a distinct category of Large language model, and competitors including DeepSeek-R1 and Google's Gemini "Thinking" variants followed within roughly a year.
Mechanisms
Test-time compute can be spent in several ways. The simplest is longer Chain-of-thought generation: the model writes out intermediate reasoning steps as text tokens before committing to an answer, and allowing more tokens generally improves performance on tasks with a clear right answer, such as arithmetic or code correctness. A second family of techniques samples multiple independent answers and selects the best one, either by majority vote (self-consistency) or by scoring candidates with a separate verifier or reward model, sometimes organized as a tree search over partial solutions. A third approach revises: the model produces a draft, critiques it, and regenerates, spending extra passes to catch errors rather than committing to the first output.
These methods differ from the In-context learning and prompting techniques that came before them mainly in who controls the extra compute. Prompting techniques like Few-shot learning examples ask a model to use its existing weights more effectively; test-time compute methods, especially in models trained with Reinforcement learning on verifiable rewards, let the model itself decide how much internal deliberation a given problem needs, in principle spending little compute on easy questions and much more on hard ones.
Economics and trade-offs
Because reasoning models can consume many times more tokens per answer than a standard chat model, test-time compute reintroduces a cost and latency trade-off that pretraining scale had mostly pushed to one side: labs and users can now buy accuracy directly, at inference time, by paying for more thinking. This has practical consequences for product design, since a coding agent or research assistant built on a reasoning model may take tens of seconds or minutes to answer a hard query, and API pricing for these models typically reflects both output length and a premium per token.
Reception and open questions
Researchers have debated how test-time compute relates to genuine reasoning versus a more expensive form of pattern matching, and results on benchmarks such as ARC-AGI have been used on both sides of that argument: reasoning models improved scores substantially over earlier generations, but performance still degrades on problems that require reasoning patterns not well represented in training data. Some labs describe test-time compute scaling as a new axis comparable in importance to the original scaling laws, with the expectation that further gains will come from more efficient search and verification rather than purely longer chains of thought. Others note that gains from added inference compute show diminishing returns past a point, and that the technique is best understood as a complement to, rather than a replacement for, pretraining scale and Fine-tuning. By 2025, test-time compute had also become central to competitive claims about Artificial general intelligence timelines, since problem-specific accuracy gains from "thinking longer" gave labs a new, easily marketed capability dial distinct from headline parameter counts.