practiceEngineering & architecture

Rate Limiting

The controlled restriction of how many operations an actor may perform within a defined scope and time window.

Download Markdown (.md)

01What it means

Rate limiting meters requests or resource use against a rule and delays or rejects activity beyond the allowance. Limits may apply per user, tenant, credential, endpoint, cost unit, or service and can use fixed windows, sliding windows, token buckets, or similar algorithms. Throttling is often used as a broader name for controlling consumption.[1][2]

Consumption control loop Meter usage, compare it with an allowance, then communicate the result
  1. 01Identify actorUser, tenant, key, endpoint
  2. 02Measure costRequests or weighted units
  3. 03Apply allowanceWindow or token bucket
  4. 04Allow, delay, rejectClear retry information
  5. 05Observe + tuneRejections and downstream health

The accountable actor, cost unit, time horizon, and algorithm determine fairness and burst behavior.

02Why it matters

Limits protect shared capacity, preserve fairness, contain accidental loops, and make commercial quotas enforceable. Because limits can block legitimate users and integrations, they are also part of the product contract and need understandable feedback and recovery behavior.

03When to use it

Use rate limits at scarce or expensive boundaries, public APIs, multi-tenant workloads, and abuse-sensitive actions. Choose scope and allowance from capacity and user needs. Rate limiting complements rather than replaces authentication, authorization, autoscaling, and denial-of-service protection.

04How it works

  1. Choose the protected resource, accountable actor, unit of cost, and time horizon.
  2. Select an algorithm whose burst and fairness behavior matches the product need.
  3. Count accepted consumption consistently across the instances enforcing the limit.
  4. Allow, delay, or reject the request and return clear limit and retry information.
  5. Separate quotas where user value or workload cost differs materially.
  6. Monitor rejection rates, saturation, false positives, bypasses, and downstream health.

05A practical example

Example

An analytics API assigns each workspace a sustained query allowance with a small burst capacity. When exceeded, it returns HTTP 429 and a retry delay. Expensive exports consume more cost units than lightweight metadata reads.

06Where it breaks

Distributed counters trade accuracy, latency, and availability. A single coarse limit can punish efficient users or fail to protect the actual bottleneck. Limits can also create synchronized retries and confusing experiences if clients lack visible remaining capacity and retry guidance.

Common mistakes

  • Applying one global request count to operations with very different costs.
  • Returning unexplained errors without a stable actor scope or retry guidance.
  • Assuming rate limiting alone provides authentication or volumetric attack protection.
  • Ignoring bursts and distributed counter behavior when choosing an algorithm.

07Sources

  1. RFC 6585: Additional HTTP Status CodesInternet Engineering Task Force · Internet standardRFC 6585: Additional HTTP Status Codes
  2. Throttling PatternMicrosoft Learn · Institutional architecture guideThrottling Pattern
View saved topics

Start typing to search titles, definitions, categories, and frameworks.