Sampling temperature is a parameter used when generating text or other content from a probabilistic model that controls how random or deterministic the output is, by rescaling the probability distribution the model samples the next token from before drawing a value. It is one of the most commonly exposed settings in APIs and interfaces for large language models and other autoregressive models.
How it works
An autoregressive language model produces, at each step, a probability distribution over possible next tokens, typically computed with a softmax function over the model's raw output scores, or logits. This process happens at Inference (AI) time, after the model's weights have been fixed by training. Temperature is applied by dividing those logits by a temperature value before the softmax step. A temperature of 1.0 leaves the distribution unchanged from what the model computed. A temperature below 1.0 sharpens the distribution, making the most likely tokens even more likely to be chosen and reducing the odds of a low-probability token appearing; a temperature approaching 0 approximates always picking the single highest-probability token, sometimes called greedy decoding. A temperature above 1.0 flattens the distribution, giving lower-probability tokens a comparatively better chance of being selected, which increases the variety and unpredictability of the output, but can also increase the chance of Hallucination (AI), since a less probable but incorrect token becomes more likely to be sampled.
Relation to other sampling controls
Temperature is usually combined with other decoding controls rather than used alone. Top-p sampling, also called nucleus sampling, restricts the model to choosing from the smallest set of tokens whose cumulative probability exceeds a threshold p, which prevents temperature from occasionally selecting an extremely unlikely token even when raised high. Top-k sampling similarly restricts sampling to a fixed number of the most likely tokens. These settings interact: a high temperature combined with a loose top-p or top-k restriction can produce text that becomes incoherent, while temperature at or near zero combined with tight restrictions produces highly repetitive, deterministic output, which is sometimes desirable for tasks like code generation or data extraction where a single, consistent answer is preferred over creative variation.
Practical use
Low or zero temperature settings are typically used for tasks that benefit from consistency and correctness, such as factual question answering, structured data extraction, mathematical calculation, and code generation, where variation in phrasing is not valuable and can introduce errors. Reasoning models that generate long chains of intermediate reasoning are often run at low temperature to keep that process stable. Higher temperature settings are used for tasks that benefit from variety, such as brainstorming, creative writing, dialogue for chatbots or game characters, and some applications of Text-to-image generation generation, where multiple distinct outputs from the same input can be useful. Because temperature directly trades off predictability against variety, it is a frequent subject of Prompt engineering guidance, and evaluation suites and benchmarks typically fix temperature at or near zero to make results reproducible across runs. Temperature does not affect a model's underlying knowledge or reasoning ability; it only changes how the model selects among the outputs its training already made plausible, which is why raising temperature can produce more varied but not more accurate answers.