termEngineering & architecture

Toil

Toil is manual, repetitive, automatable operational work that creates little enduring value and grows with the service it supports.

Download Markdown (.md)

01What it means

In site reliability engineering, toil is production-related work that tends to be manual, repetitive, automatable, tactical, without enduring improvement, and proportional to service growth. Examples include repeatedly applying routine fixes, handling predictable tickets, and executing runbook steps by hand. Unpleasant work is not automatically toil, and valuable one-time cleanup may not be toil at all.[1][2]

Recurring human effort that scales with the service The visible task repeats because the underlying system never improves
Visible effectManual operational task

Routine fix, predictable ticket, repeated runbook step

Repetitive + tacticalSame response without lasting change

AutomatableRules and inputs are sufficiently repeatable

Growth-proportionalHuman effort rises with service demand

No enduring improvementCompletion does not reduce the next occurrence

Unpleasant work is not automatically toil, and valuable one-time cleanup may create enduring improvement rather than recurring burden.

02Why it matters

Unchecked toil consumes the capacity needed to improve reliability and product capability. As a service grows, work that scales linearly with customers, machines, or incidents can overwhelm a team, increase interruption and error, and contribute to burnout. Naming and measuring toil turns a vague complaint into an engineering and operational investment question.

03When to use it

Use the concept when reviewing operational load, on-call work, support queues, deployment processes, access requests, or recurring maintenance. Track enough detail to identify patterns and prioritize reduction. Do not automate blindly: rare work requiring judgment, unsafe automation, or a process that should be removed entirely may call for a different response.

04How it works

  1. Capture recurring operational tasks, their frequency, hands-on time, interruption cost, risk, and growth driver.
  2. Assess each task against toil characteristics rather than labeling all support or maintenance work undesirable.
  3. Prioritize candidates using cumulative cost, error risk, morale, and the expected return from intervention.
  4. Remove the underlying need where possible; otherwise simplify, enable self-service, partially automate, or fully automate with safeguards.
  5. Measure whether the intervention reduced human load and failure, then prevent new toil through design and operational review.

05A practical example

Example

An operations team spends fifteen hours each week manually provisioning standard sandbox accounts from tickets. It first removes unnecessary approvals, then offers a validated self-service workflow with quotas and audit logs. The intervention reduces queue time and recurring effort; merely writing a faster runbook would have left most toil intact.

06Where it breaks

Toil measurement can become toil itself, and teams may disagree about what has enduring value. Automation requires maintenance and can amplify mistakes at scale. Some manual work is temporarily rational at low volume, while aggressive reduction can remove useful contact with customers or production systems. The economic and safety case should determine the response.

Common mistakes

  • Calling every disliked, administrative, or operational task toil.
  • Automating a broken process before asking whether it can be eliminated or simplified.
  • Celebrating automation output without measuring reduced human effort, error, or service risk.

07Sources

  1. Eliminating ToilGoogle Site Reliability Engineering · Foundational book chapterEliminating Toil
  2. Operational Efficiency: Eliminating ToilGoogle Site Reliability Engineering Workbook · Institutional practice guideOperational Efficiency: Eliminating Toil
View saved topics

Start typing to search titles, definitions, categories, and frameworks.