metricEngineering & architecture

Latency

Latency measures elapsed time between a defined start and response point, exposing the delays users experience while a system works.

Download Markdown (.md)

01What it means

Latency is the time taken for an operation or response across a specified boundary. Network transit, server processing, queue waiting, and complete user interactions have different boundaries and should be named. It differs from throughput, which counts work per unit time, and bandwidth, which describes a communication link’s capacity.[1][2]

02Why it matters

A system can process many requests while some users wait a long time. Latency distributions show whether slow experiences are concentrated in particular regions, endpoints, or operations that averages obscure.

03When to use it

Use it when defining response expectations or investigating slow tasks. Select a boundary relevant to the user and report percentiles and sample counts. Server timing alone is insufficient when network or browser work dominates the experience.

04How it works

  1. Define the operation, start event, completion event, and measurement location.
  2. Collect durations for representative traffic and separate success from failure.
  3. Report the distribution, including relevant percentiles and traffic volume.
  4. Segment by task, region, device, or dependency to locate delays.
  5. Compare improvements with user-facing goals under comparable demand.

05A practical example

Example

A search API averages 180 milliseconds but its p95 reaches two seconds for one region. The team checks regional traces rather than adding a global loading animation and finds an avoidable cross-region dependency.

06Where it breaks

Percentiles are estimates influenced by sampling and measurement windows; averaging percentiles from separate groups does not produce a valid combined percentile. Fast failures can make an overall average look better while service quality worsens. Synthetic timing may differ from real-user timing.

Common mistakes

  • Reporting only the average.
  • Comparing server processing time with full browser response time.
  • Combining successful and failed request timing without explaining the mix.

07Sources

  1. Monitoring Distributed SystemsGoogle SRE · Engineering handbookMonitoring Distributed Systems
  2. What is Latency?Amazon Web Services · Technical guidanceWhat is Latency?
View saved topics

Start typing to search titles, definitions, categories, and frameworks.