Data-Informed Decision Making
Data-informed decision making combines relevant evidence with judgment, context, values, constraints, and accountability instead of letting a metric decide.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/data-informed-decision-making/
01What it means
Data-informed decision making is a practice of using the best available quantitative and qualitative evidence to improve a decision while recognizing that evidence is incomplete. Decision makers also consider strategy, user needs, ethics, feasibility, uncertainty, and consequences, then remain accountable for the choice.[1][2][3]
- Behavioral data
- Recorded actions and system conditions
- Qualitative evidence
- Needs, meanings, mechanisms, and lived experience
- Business context
- Strategy, economics, timing, and capability
- Values + duties
- Rights, safety, fairness, and long-term effects
- Uncertainty
- Gaps, bias, assumptions, and alternative explanations
- Judgment
- Explicit tradeoff, owner, and revision trigger
Metrics are partial representations. A responsible decision states uncertainty, alternatives, affected people, and what evidence would trigger revision.
02Why it matters
Product decisions occur under uncertainty, and intuition alone is vulnerable to selective memory and organizational power. Data can challenge assumptions and estimate outcomes, while context prevents teams from treating an imperfect proxy as truth. The combination supports decisions that are explainable, revisable, and connected to evidence.
03When to use it
Use this practice for prioritization, launches, experiment reviews, strategy, resource allocation, and operational changes. Match the strength of evidence to the decision’s reversibility and risk. High-impact or hard-to-reverse choices deserve broader evidence, stronger validation, and explicit ethical review.
04How it works
- Frame the decision, alternatives, affected groups, constraints, timing, and criteria before inspecting convenient metrics.
- Gather relevant behavioral, operational, financial, qualitative, and external evidence; document gaps and provenance.
- Assess data quality, uncertainty, representativeness, incentives, privacy, and plausible alternative explanations.
- Compare options using evidence plus strategy, values, expertise, and likely second-order effects.
- Record the decision and assumptions, define signals that would change it, and review outcomes after action.
05A practical example
A team considers removing a rarely used accessibility setting. Usage events show low activation, but interviews reveal that a small group depends on it to complete critical tasks and that discovery is poor. The team improves findability and measurement instead of interpreting low aggregate use as low value.
06Where it breaks
More data does not eliminate uncertainty or value judgments. Available evidence may systematically exclude people, and analysis can be used to rationalize a preferred answer. Decision speed can suffer when teams demand impossible certainty, while dashboards can falsely imply neutrality.
Common mistakes
- Using “the data says” to hide assumptions and ownership.
- Considering only evidence that is easy to quantify.
- Failing to define what future evidence would reverse the decision.
07Sources
- Evidence, Policy, Impact: WHO Guide for Evidence-Informed Decision-MakingWorld Health Organization · Institutional evidence guideEvidence, Policy, Impact: WHO Guide for Evidence-Informed Decision-Making
- Data Competency FrameworkCanada School of Public Service · Government competency frameworkData Competency Framework
- Measuring the User Experience on a Large ScaleGoogle Research · Peer-reviewed product measurement paperMeasuring the User Experience on a Large Scale