# Formative and Summative Evaluation

A distinction between evaluation used to improve a design and evaluation used to judge overall performance against a reference point.

- Type: framework
- Domain: Research & behavior
- Level: Intermediate
- Last reviewed: 2026-09-25
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/formative-and-summative-evaluation/>)

## What it means

Formative evaluation produces evidence for improving a product or service during design and iteration. Summative evaluation assesses overall performance at a defined point, often against an earlier version, competitor, benchmark, or release criterion. The distinction concerns the purpose and timing of evaluation, not whether the evidence is qualitative or quantitative.

## Why it matters

Confusing the two purposes creates mismatched studies. An exploratory usability round can reveal what to repair but cannot provide a stable benchmark, while a tightly standardized benchmark may quantify performance without explaining how to improve it. Naming the purpose clarifies methods, sample needs, measures, and decisions.

## When to use it

Use formative evaluation while shaping or revising a design and summative evaluation when a team needs a defensible comparison or overall assessment. A program may use both in sequence, and a summative study can still uncover formative observations, but the primary decision should determine the design.

## How it works

1. State whether the decision is to improve the design or assess its overall performance.
2. For formative work, choose methods and participants that expose causes, breakdowns, and possible changes.
3. Iterate the design and repeat evaluation as the most consequential problems are addressed.
4. For summative work, define stable tasks, measures, comparison points, sampling, and analysis in advance.
5. Report conclusions within the limits of the design and carry newly discovered issues into the next formative cycle.

## A practical example

A travel team runs small prototype sessions to learn why passengers miss the baggage step and revises the flow twice. Before launch, it runs a standardized study with predefined success, error, and time measures against the current product. The first studies are formative; the controlled comparison is summative.

## Where it breaks

The labels do not determine method quality, sample size, or whether a study is qualitative. A premature summative score can freeze an immature design, while endless formative rounds can postpone a necessary performance decision. Comparisons are only meaningful when tasks, participants, conditions, and measures are sufficiently compatible.

## Common mistakes

- Equating formative with qualitative and summative with quantitative.
- Using a small diagnostic study to claim population-level improvement.
- Running a benchmark without a stable protocol or meaningful comparison point.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/formative-and-summative-evaluation/#visualization-formative-and-summative-evaluation>)

## Sources

1. [Formative vs. Summative Evaluations](<https://www.nngroup.com/articles/formative-vs-summative-evaluations/>) — Nielsen Norman Group · Professional guidance
2. [When to Use Which User-Experience Research Methods](<https://www.nngroup.com/articles/which-ux-research-methods/>) — Nielsen Norman Group · Professional guidance

## Related topics

- [Usability Testing](<https://wiki.soroush.uk/topic/usability-testing/>)
- [Usability Benchmarking](<https://wiki.soroush.uk/topic/usability-benchmarking/>)
- [Generative and Evaluative Research](<https://wiki.soroush.uk/topic/generative-and-evaluative-research/>)
