termEngineering & architecture

Load Balancer

A component that distributes incoming traffic across eligible service instances or locations.

Download Markdown (.md)

01What it means

A load balancer receives traffic through a stable address and selects one of several backend targets using health, capacity, location, connection, or scheduling information. Load balancing can operate at transport or application layers and at local or global scope. It distributes work; it does not create healthy capacity by itself.[1][2]

Stable entry, changing healthy capacity The load balancer selects an eligible backend for each unit of traffic
Stable listener / load balancer
Client traffic
Connections or requests
Health checks
Safe-to-serve signal
Routing policy
Weight, locality, latency, capacity
Backend A
Eligible target
Backend B
Eligible target
Drain + remove
Deployments and failures

Routing uses health and scheduling information; load balancing distributes work but does not create healthy capacity.

02Why it matters

Distributing requests lets services use several instances, replace unhealthy targets, scale horizontally, and shift traffic during releases or regional disruption. The balancing policy directly affects latency, session behavior, utilization, and the size of failures users experience.

03When to use it

Use a load balancer when interchangeable service instances should share traffic or when routing must account for health and location. It adds little value for a single fixed target, and stateful workloads may require session, data-locality, or connection-aware designs beyond simple round robin.

04How it works

  1. Expose a stable listener and register eligible backend targets or locations.
  2. Run health checks that reflect whether each target can safely serve traffic.
  3. Choose a routing algorithm based on connections, weight, locality, latency, or capacity.
  4. Forward traffic while preserving the identity and protocol context the application needs.
  5. Drain targets during deployments and remove unhealthy instances from selection.
  6. Monitor target saturation, imbalance, health-check quality, errors, and added latency.

05A practical example

Example

A collaboration service runs six stateless API instances across three zones. The load balancer removes one instance after failed health checks and drains two old instances during deployment, keeping existing connections stable while new traffic moves elsewhere.

06Where it breaks

Poor health checks can route traffic to broken targets or remove healthy ones. Uneven or stateful workloads may remain imbalanced, and the balancer itself needs capacity and redundancy. Distribution cannot compensate for a shared database bottleneck or universally broken release.

Common mistakes

  • Treating load balancing as automatic scalability without provisioning backend capacity.
  • Using shallow health checks that pass while critical dependencies are unusable.
  • Ignoring long-lived connections, session affinity, or graceful connection draining.
  • Assuming equal request counts mean equal resource use.

07Sources

  1. What Is Elastic Load Balancing?Amazon Web Services · Official technical documentationWhat Is Elastic Load Balancing?
  2. Cloud Load Balancing OverviewGoogle Cloud · Official technical documentationCloud Load Balancing Overview
View saved topics

Start typing to search titles, definitions, categories, and frameworks.