principleData, experimentation & growth

Algorithmic Fairness

Algorithmic fairness evaluates and addresses unjust differences in how automated systems allocate benefits, impose burdens, or represent people.

Download Markdown (.md)

01What it means

Algorithmic fairness concerns the treatment and outcomes of people affected by algorithmic systems. Assessment may compare selection rates, error rates, or prediction quality across relevant groups, but a fairness metric encodes a particular concern rather than a universal definition of justice. Harm can arise from the task, labels, data, model, interface, or surrounding institution, even without discriminatory intent.[1][2]

Fairness depends on the harm and institutional context Inspect disparities across the task, data, model, workflow, and recourse
  1. 01Task + institutionShould this decision be automated, and who bears its burdens?
  2. 02Data + labelsSampling, missingness, proxies, history, and reference validity
  3. 03Model behaviorSelection and error patterns across relevant intersections
  4. 04WorkflowHow outputs, thresholds, and human reliance affect outcomes
  5. 05RecourseExplanation, correction, appeal, and remedy
  6. 06MonitoringChanging populations, impacts, and tradeoffs over time

No single metric represents justice. Selection rates, error rates, and prediction quality encode different concerns and can conflict.

02Why it matters

Good average performance can conceal repeated exclusion or unequal errors. Product teams need to identify who benefits, who bears mistakes, and which disparities matter before choosing an optimization objective or release threshold.

03When to use it

Assess fairness when automated ranking, allocation, classification, or representation affects people's opportunities or experiences. Involve affected groups and appropriate specialists when defining harms and handling sensitive attributes. Use qualitative investigation alongside quantitative measures when impacts cannot be captured by available labels.

04How it works

  1. Define the decision, affected groups, potential harms, and relevant context with stakeholders.
  2. Examine how labels, sampling, missing data, proxies, and workflow choices may create or conceal disparities.
  3. Choose measures tied to the harm, such as selection-rate differences or false-positive and false-negative rates, with uncertainty estimates.
  4. Compare groups and meaningful intersections while checking small samples and the validity of the reference labels.
  5. Evaluate changes to data, model, thresholds, or workflow, document tradeoffs, and monitor outcomes and recourse.

05A practical example

Example

A community platform finds that a moderation classifier incorrectly removes posts in one dialect more often than others. Reviewers inspect the labels and context, the team expands evaluation coverage, and it tests changes to the model and appeal process rather than optimizing only aggregate accuracy.

06Where it breaks

Different fairness criteria can conflict, and equal measured rates do not prove the absence of harm. Group definitions and historical labels can be contested or incomplete. Sensitive data requires careful governance, and small samples can make apparent differences unstable.

Common mistakes

  • Declaring a system fair because protected attributes were removed while proxies remain.
  • Selecting whichever fairness metric makes current results look best.
  • Treating historical human decisions as unbiased ground truth.
  • Ignoring intersectional groups, uncertainty, or harms outside the measured decision.

07Sources

  1. Towards a Standard for Identifying and Managing Bias in Artificial IntelligenceNational Institute of Standards and Technology · Government research reportTowards a Standard for Identifying and Managing Bias in Artificial Intelligence
  2. Common fairness metricsFairlearn · Official documentationCommon fairness metrics
View saved topics

Start typing to search titles, definitions, categories, and frameworks.