practiceData, experimentation & growth

Data-Informed Decision Making

Data-informed decision making combines relevant evidence with judgment, context, values, constraints, and accountability instead of letting a metric decide.

Download Markdown (.md)

01What it means

Data-informed decision making is a practice of using the best available quantitative and qualitative evidence to improve a decision while recognizing that evidence is incomplete. Decision makers also consider strategy, user needs, ethics, feasibility, uncertainty, and consequences, then remain accountable for the choice.[1][2][3]

Evidence informs judgment without becoming the decision maker Quantitative and qualitative evidence meet context, values, constraints, and accountability
Accountable product decision
Behavioral data
Recorded actions and system conditions
Qualitative evidence
Needs, meanings, mechanisms, and lived experience
Business context
Strategy, economics, timing, and capability
Values + duties
Rights, safety, fairness, and long-term effects
Uncertainty
Gaps, bias, assumptions, and alternative explanations
Judgment
Explicit tradeoff, owner, and revision trigger

Metrics are partial representations. A responsible decision states uncertainty, alternatives, affected people, and what evidence would trigger revision.

02Why it matters

Product decisions occur under uncertainty, and intuition alone is vulnerable to selective memory and organizational power. Data can challenge assumptions and estimate outcomes, while context prevents teams from treating an imperfect proxy as truth. The combination supports decisions that are explainable, revisable, and connected to evidence.

03When to use it

Use this practice for prioritization, launches, experiment reviews, strategy, resource allocation, and operational changes. Match the strength of evidence to the decision’s reversibility and risk. High-impact or hard-to-reverse choices deserve broader evidence, stronger validation, and explicit ethical review.

04How it works

  1. Frame the decision, alternatives, affected groups, constraints, timing, and criteria before inspecting convenient metrics.
  2. Gather relevant behavioral, operational, financial, qualitative, and external evidence; document gaps and provenance.
  3. Assess data quality, uncertainty, representativeness, incentives, privacy, and plausible alternative explanations.
  4. Compare options using evidence plus strategy, values, expertise, and likely second-order effects.
  5. Record the decision and assumptions, define signals that would change it, and review outcomes after action.

05A practical example

Example

A team considers removing a rarely used accessibility setting. Usage events show low activation, but interviews reveal that a small group depends on it to complete critical tasks and that discovery is poor. The team improves findability and measurement instead of interpreting low aggregate use as low value.

06Where it breaks

More data does not eliminate uncertainty or value judgments. Available evidence may systematically exclude people, and analysis can be used to rationalize a preferred answer. Decision speed can suffer when teams demand impossible certainty, while dashboards can falsely imply neutrality.

Common mistakes

  • Using “the data says” to hide assumptions and ownership.
  • Considering only evidence that is easy to quantify.
  • Failing to define what future evidence would reverse the decision.

07Sources

  1. Evidence, Policy, Impact: WHO Guide for Evidence-Informed Decision-MakingWorld Health Organization · Institutional evidence guideEvidence, Policy, Impact: WHO Guide for Evidence-Informed Decision-Making
  2. Data Competency FrameworkCanada School of Public Service · Government competency frameworkData Competency Framework
  3. Measuring the User Experience on a Large ScaleGoogle Research · Peer-reviewed product measurement paperMeasuring the User Experience on a Large Scale
View saved topics

Start typing to search titles, definitions, categories, and frameworks.