# Message Queue

A durable buffer from which consumers claim messages, allowing producers and workers to operate at different times and rates.

- Type: term
- Domain: Engineering & architecture
- Level: Intermediate
- Last reviewed: 2026-09-24
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/message-queue/>)

## What it means

A message queue stores units of work or communication until a consumer receives and processes them. Producers enqueue messages without waiting for the work to finish; one of one or more competing consumers typically handles each message and acknowledges completion. Delivery and ordering guarantees vary by queue.

## Why it matters

Queues can keep slow or bursty work out of user-facing requests, absorb temporary demand spikes, and allow workers to scale independently. They also make backlog age, duplicate delivery, poison messages, and eventual completion part of the product's behavior.

## When to use it

Use a queue when work can complete asynchronously, producers and consumers need temporal decoupling, or demand must be buffered. Avoid it when the caller requires an immediate result, the operation is trivial, or the team cannot operate retries, idempotency, monitoring, and recovery.

## How it works

1. A producer writes a message with the minimum data and identifiers needed to perform the work.
2. The queue stores the message durably and exposes it to an eligible consumer.
3. A consumer claims the message, processes it, and acknowledges successful completion.
4. Unacknowledged work becomes visible again or is retried according to delivery policy.
5. Repeatedly failing messages move to a failure destination for investigation and recovery.
6. Operators track backlog size, oldest-message age, processing time, failures, and business completion.

## A practical example

An image service places each uploaded file on a resize queue and responds immediately with a processing state. Workers create variants at a controlled rate; duplicate messages are safe, and jobs that repeatedly fail move to a dead-letter queue for review.

## Where it breaks

Queues add infrastructure, latency, and failure states. Common systems provide at-least-once delivery, so duplicates and reordering are normal possibilities. A growing backlog can hide overload, and accepting a message does not mean the business operation completed.

## Common mistakes

- Assuming a queued message will be processed exactly once or in global order.
- Acknowledging before durable business work finishes and losing failed work.
- Sending large, unstable internal objects instead of a clear message contract.
- Monitoring queue availability but not backlog age or end-to-end completion.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/message-queue/#visualization-message-queue>)

## Sources

1. [What Is Amazon Simple Queue Service?](<https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html>) — Amazon Web Services · Official technical documentation
2. [Queue-Based Load Leveling Pattern](<https://learn.microsoft.com/en-us/azure/architecture/patterns/queue-based-load-leveling>) — Microsoft Learn · Institutional architecture guide

## Related topics

- [Background Job](<https://wiki.soroush.uk/topic/background-job/>)
- [Event-Driven Architecture](<https://wiki.soroush.uk/topic/event-driven-architecture/>)
- [Idempotency](<https://wiki.soroush.uk/topic/idempotency/>)
