# Sampling Temperature

Sampling temperature adjusts how strongly a generative model favors higher-scoring next tokens when sampling an output.

- Type: term
- Domain: Engineering & architecture
- Level: Intermediate
- Last reviewed: 2026-10-05
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/sampling-temperature/>)

## What it means

Temperature is a decoding parameter that rescales token scores before sampling. Lower positive values concentrate probability on favored tokens; higher values spread it more broadly. Some interfaces expose zero as a special setting for greedy selection. The term describes output sampling, not factual confidence, and differs from temperature scaling used to calibrate predictive probabilities.

## Why it matters

The setting affects variation between responses and can change task quality. A writing assistant may benefit from varied candidates, while extraction requires consistent behavior. Neither a low setting nor repeated agreement establishes that an answer is correct.

## When to use it

Tune temperature when the chosen model exposes the control and evaluation shows that response variation matters. Follow model-specific guidance: supported ranges, defaults, and recommended settings vary. Use validation and deterministic code for guarantees rather than relying on a sampling parameter.

## How it works

1. Define whether the task values variation, repeatability, or another measurable quality.
2. Record the model version, prompt, supported temperature range, and other decoding controls.
3. Compare a small set of settings across representative inputs, collecting multiple responses for each.
4. Assess correctness, repetition, useful diversity, and downstream failures rather than judging writing style alone.
5. Deploy the evaluated configuration and retest when model versions or task requirements change.

## A practical example

A product team generates alternative onboarding headlines. It compares several sampling settings on the same briefs, reviewing whether candidates remain accurate and distinct. The selected setting provides useful alternatives, while a separate check rejects unsupported promises; temperature is not used as that check.

## Where it breaks

The same numeric setting is not directly comparable across models. Top-p filtering and other decoding choices interact with sampling. Greedy selection can still vary across serving conditions, and reducing variation can preserve a consistently wrong answer.

## Common mistakes

- Describing temperature as a creativity percentage or a truthfulness score.
- Promising identical outputs merely because temperature is zero.
- Changing several decoding settings together without isolating effects.
- Confusing sampling temperature with probability calibration.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/sampling-temperature/#visualization-sampling-temperature>)

## Sources

1. [Generation](<https://huggingface.co/docs/transformers/main_classes/text_generation>) — Hugging Face · Official documentation
2. [Content generation parameters](<https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/content-generation-parameters>) — Google Cloud · Official documentation

## Related topics

- [Large Language Model \(LLM\)](<https://wiki.soroush.uk/topic/large-language-model/>)
- [Probability Calibration](<https://wiki.soroush.uk/topic/probability-calibration/>)
- [Model Evaluation](<https://wiki.soroush.uk/topic/model-evaluation/>)
