Counter Metrics
Counter metrics test whether improving a focal metric is producing an opposing cost, distorted behavior, or unintended decline elsewhere.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/counter-metrics/
01What it means
A counter metric is a deliberately paired measure that challenges the interpretation of a target or success metric. It captures a plausible negative effect, competing outcome, or quality loss that could accompany apparent improvement. In many contexts it overlaps with a balancing or guardrail measure.[1][2]
- Focal metric
- The outcome being optimizedState how an optimizer could increase it
- Counter metric
- Competing outcome, quality loss, or delayed costMeasure who or what could bear the downside
Interpret the pair together and revise it when teams discover new optimization paths or system effects.
02Why it matters
Any single metric offers an incomplete view and can become easier to game when promoted as a target. Counter metrics force teams to ask what might worsen if the focal number rises. They support more honest decisions by showing whether success is merely displaced cost.
03When to use it
Use counter metrics when setting targets, designing dashboards, evaluating experiments, or defining incentives. They are especially important for engagement, speed, automation, and conversion goals, where aggressive optimization can harm quality, trust, accessibility, or downstream workload.
04How it works
- State the focal metric and describe the behavior an optimizer might use to increase it.
- Identify who or what could bear the cost, including another journey stage, user group, team, or future period.
- Choose one or more measures that would reveal those costs with appropriate segmentation and timing.
- Define acceptable ranges and interpret the focal and counter metrics as a set.
- Revise the pair when teams discover new gaming paths or system effects.
05A practical example
A support product aims to reduce median ticket resolution time. Its counter metrics include seven-day reopen rate, customer-reported resolution, escalations, and unresolved-ticket age. Faster closure accompanied by more reopened tickets is treated as degraded service, not operational improvement.
06Where it breaks
Counter metrics are selected from known risks and may miss novel harms. Too many can paralyze decisions, while poorly matched measures create noise. They do not prove that one metric caused another to move, and a stable average may still conceal harm in a segment.
Common mistakes
- Pairing a target only with another metric that rewards the same behavior.
- Selecting counter metrics after incentives have already produced gaming.
- Using aggregate values without checking affected populations.
07Sources
- Establishing MeasuresInstitute for Healthcare Improvement · Institutional improvement guidanceEstablishing Measures
- Measuring the User Experience on a Large ScaleGoogle Research · Peer-reviewed framework paperMeasuring the User Experience on a Large Scale