# Automation Bias

Automation bias is the tendency to rely on automated advice in ways that cause people to miss problems or accept incorrect recommendations.

- Type: term
- Domain: Research & behavior
- Level: Intermediate
- Last reviewed: 2026-09-26
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/automation-bias/>)

## What it means

Automation bias occurs when automated output displaces appropriate attention to other evidence. It can cause commission errors, such as following a wrong recommendation, and omission errors, such as overlooking a problem because the system did not flag it. It predates generative AI. Reliance is not inherently biased: the concern is reliance that exceeds what the system's reliability and task context justify.

## Why it matters

A human approval step can fail even when reviewers are competent. Defaults, time pressure, workload, and apparently authoritative advice influence whether people verify a result. Teams should measure the combined human-system outcome, particularly when the automation is wrong.

## When to use it

Consider automation bias in recommendation, moderation, ranking, decision-support, and approval workflows. Test with realistic workloads and occasional incorrect suggestions. Avoid adding friction indiscriminately: extra verification should improve decisions enough to justify its effort.

## How it works

1. Identify where automated suggestions influence human decisions and which independent evidence is available.
2. Observe decisions with and without advice, including controlled cases where the advice is wrong or incomplete.
3. Measure acceptance of incorrect suggestions and missed issues separately from overall completion speed.
4. Test interventions such as independent initial judgment, accessible source evidence, reversible actions, and selective review prompts.
5. Reassess error detection, workload, accessibility, and appropriate reliance over repeated use.

## A practical example

A moderation queue preselects an AI recommendation. In a controlled study, reviewers accept several incorrect removals despite contradictory context. The team tests requiring an initial judgment for ambiguous cases and compares corrected errors and review time before changing the workflow.

## Where it breaks

Observed reliance also reflects expertise, incentives, task difficulty, and system quality. Findings from one setting may not transfer to another. Cognitive forcing interventions can slow work or frustrate users, and explanations sometimes encourage unwarranted confidence rather than better judgment.

## Common mistakes

- Blaming reviewers without examining workload, defaults, and evidence access.
- Treating every accepted recommendation as successful assistance.
- Adding a generic warning while leaving the decision environment unchanged.
- Assuming explanations or a confirmation button eliminate overreliance.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/automation-bias/#visualization-automation-bias>)

## Sources

1. [Decision Making In A High-Tech World: Automation Bias and Countermeasures](<https://ntrs.nasa.gov/citations/20020041010>) — NASA Technical Reports Server · Research report
2. [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](<https://arxiv.org/abs/2102.09692>) — ACM · Peer-reviewed research

## Related topics

- [Human in the Loop \(HITL\)](<https://wiki.soroush.uk/topic/human-in-the-loop/>)
- [Human–AI Interaction](<https://wiki.soroush.uk/topic/human-ai-interaction/>)
- [Cognitive Load](<https://wiki.soroush.uk/topic/cognitive-load/>)
