Vanity and Actionable Metrics
Vanity and actionable metrics distinguish impressive-looking counts from measures that are clearly defined, decision-linked, and capable of informing responsible action.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/vanity-and-actionable-metrics/
01What it means
A vanity metric is a number that may look favorable but does not reliably guide a decision, often because it is cumulative, context-free, or easily inflated. An actionable metric is tied to a defined behavior, population, time window, owner, and decision; the distinction is contextual, so the same measure can be useful or vanity depending on its use.[1][2]
02Why it matters
Teams can optimize visible totals while missing activation, retention, quality, or harm. Requiring a decision link turns reporting into learning and makes it harder to use growth in a headline number as evidence that a product strategy works.
03When to use it
Use this framework while defining dashboards, KPIs, experiments, and reviews. Keep broad awareness measures when they serve a legitimate communication purpose, but pair them with rates, cohorts, and guardrails when making operational or product choices.
04How it works
- Name the decision and accountable owner for each metric.
- Define population, denominator, time window, source, and refresh cadence.
- Connect movement to a plausible action and expected outcome.
- Add diagnostic and guardrail measures that reveal tradeoffs.
- Retire or relabel measures that no decision uses or that are easily gamed.
05A practical example
A product celebrates total registered accounts, but new registrations include duplicates and inactive users. The team retains the total for reporting while using verified first-value completion and four-week retention to evaluate onboarding changes.
06Where it breaks
No metric is inherently actionable, and an overemphasis on immediate action can neglect long-term brand, trust, or discovery signals. Better metrics still require judgment, qualitative evidence, and causal caution.
Common mistakes
- Labeling a metric vanity merely because it is high-level.
- Calling a measure actionable without naming the decision it changes.
- Optimizing one metric without guardrails for quality or harm.
07Sources
- Lean AnalyticsO'Reilly Media · Authoritative bookLean Analytics
- The Lean StartupThe Lean Startup · Original authorThe Lean Startup