# Algorithmic Feedback Loop

An algorithmic feedback loop occurs when a system's outputs influence the behavior or observations later used to update or evaluate that system.

- Type: term
- Domain: Data, experimentation & growth
- Level: Advanced
- Last reviewed: 2026-10-06
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/algorithmic-feedback-loop/>)

## What it means

An algorithmic feedback loop connects a model's decisions to its future evidence. A recommendation changes exposure, exposure changes recorded interactions, and those interactions shape later recommendations. The loop can be direct or pass through other products and human decisions. Algorithmic confounding describes a related problem: observations shaped by earlier recommendations are treated as if they independently revealed user preferences.

## Why it matters

More data does not necessarily provide more independent evidence. A product can repeatedly confirm its own choices, restrict exposure for new items, or amplify existing disparities while its offline metrics appear to improve. Teams need to understand how the system helped produce its training and evaluation data.

## When to use it

Analyze feedback loops when predictions affect what gets shown, investigated, labeled, or acted upon before data is reused. The concept applies to ranking, moderation, routing, and other decision systems. It is distinct from a growth loop, which describes how outputs generate further participation, and from drift caused by external changes.

## How it works

1. Map the path from model output to product action, user response, recorded data, and the next model update.
2. Identify whose outcomes become observable and which alternatives remain unseen because of the current policy.
3. Record exposure and policy versions so later analysis can distinguish model-mediated observations from other evidence.
4. Where appropriate, preserve comparison data through controlled exploration, independent review samples, or experiments with explicit assumptions.
5. Monitor exposure concentration, subgroup outcomes, and long-term utility, then evaluate interventions across multiple update cycles.

## A practical example

A marketplace ranker favors sellers with more purchases. Those sellers receive more impressions, generating still more purchases for the next training cycle. The team logs impressions and tests limited exposure for eligible new sellers, assessing buyer satisfaction and seller coverage before interpreting sales history as evidence of intrinsic quality.

## Where it breaks

A feedback loop is not inherently harmful; learning from outcomes can improve a product. Its effects depend on incentives, data selection, and update rules. Simulations reveal possible mechanisms but do not establish production impact. Breaking one loop can shift exposure or costs elsewhere, and independent evidence may be expensive.

## Common mistakes

- Assuming observed engagement represents preferences independently of what the system displayed.
- Retraining more frequently without checking whether the new data repeats the same selection bias.
- Treating every change in input distributions as an externally caused drift problem.
- Claiming bias is corrected from one aggregate metric or a single update cycle.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/algorithmic-feedback-loop/#visualization-algorithmic-feedback-loop>)

## Sources

1. [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](<https://arxiv.org/abs/1710.11214>) — ACM Conference on Recommender Systems · Peer-reviewed research
2. [Hidden Technical Debt in Machine Learning Systems](<https://research.google/pubs/hidden-technical-debt-in-machine-learning-systems/>) — Advances in Neural Information Processing Systems · Peer-reviewed research

## Related topics

- [Recommender System](<https://wiki.soroush.uk/topic/recommender-system/>)
- [Model Drift](<https://wiki.soroush.uk/topic/model-drift/>)
- [Causal Inference](<https://wiki.soroush.uk/topic/causal-inference/>)
