Few-Shot and Zero-Shot Prompting
Few-shot prompting supplies task demonstrations in a model's input, while zero-shot prompting asks for the task without demonstrations.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/few-shot-and-zero-shot-prompting/
01What it means
These prompting modes differ in whether the request includes examples of the desired task behavior. Zero-shot prompts can still contain detailed instructions and reference material. Few-shot prompts include a small set of demonstrations, often input–output pairs; one-shot uses one. This is in-context adaptation: the examples influence the current response without updating model weights, unlike fine-tuning.[1][2]
- Zero-shot prompting
- Instruction + task inputLower context cost; depends on clear task description and model capability
- Few-shot prompting
- Instruction + demonstrations + task inputCan clarify format and distinctions; example selection and ordering influence output
- Fine-tuning
- Training examples update model parametersA separate adaptation method, not simply many examples in one prompt
Examples consume context and can anchor unwanted patterns. Neither approach changes model parameters or guarantees correctness outside represented cases.
02Why it matters
Examples can communicate subtle label boundaries, formatting, and tone more clearly than additional rules. They also consume context and can teach accidental patterns. Teams need to know whether demonstrations improve representative tasks enough to justify their maintenance and runtime cost.
03When to use it
Use demonstrations when a clear instruction baseline repeatedly misses a convention that examples can illustrate. Start without examples for straightforward tasks. Missing knowledge requires appropriate evidence, and mandatory output structure may require schema enforcement even when examples are provided.
04How it works
- Define the task, acceptable outputs, and a separate evaluation set covering important user groups.
- Establish a zero-shot baseline using clear instructions and the same model configuration used for comparisons.
- Select a few correct demonstrations spanning common cases and meaningful boundaries, with consistent labels and formatting.
- Compare results on unseen cases and vary example selection or order to reveal sensitivity.
- Version the demonstration set and retain it only when quality gains justify added context, latency, and upkeep.
05A practical example
A feedback tool must distinguish a bug report from a feature request. Instructions alone misclassify requests that mention a workaround. The team adds contrasting examples and evaluates on unseen feedback, including ambiguous cases routed for review. It retains the examples only if the improvement extends beyond wording similar to the demonstrations.
06Where it breaks
Results depend on the model, task, example choice, and ordering. Demonstrations can encode bias or reveal sensitive data. Success with a few examples does not show that the model learned a durable capability or will generalize to a changed task.
Common mistakes
- Putting evaluation answers into demonstrations and reporting the resulting score as generalization.
- Calling reference documents few-shot examples when they do not demonstrate the task.
- Adding many repetitive examples without testing their marginal value.
- Treating example-based formatting as a guaranteed output contract.
07Sources
- Include few-shot examplesGoogle Cloud · Official documentationInclude few-shot examples
- Language Models are Few-Shot LearnersNeurIPS · Peer-reviewed researchLanguage Models are Few-Shot Learners