Backpressure

Pradeep P
4 days ago
3 min read

Series: Modern System Design · Layer 3 — Reliability

Layer 3 · Post 8 of 14

← Previous: Rate Limiting → Next: Replication

Layer 3 — Reliability · Post 33 of 88

Backpressure is how a busy system says 'slow down' to producers so queues do not grow without bound.

What you'll learn

Unbounded queues as a hidden outage (memory, lag, then death)
Push vs pull vs "drop / reject / block" — three ways to push back
How this differs from rate limiting (policy vs "I am full right now")

The idea in one minute

Backpressure is a signal from consumer to producer: slow down, I cannot keep up.

Producers --> [ bounded queue ] --> Consumers | full v reject / block / drop (do not grow RAM forever)

Rate limiting says "you may send 100/min." Backpressure says "even if you are under quota, this buffer is full." TCP already does this (window size). Many app designs forget and use an unbounded in-memory queue.

Why it matters

A queue that only grows looks like reliability: "we never drop work." Then the heap dies, Kafka disk fills, or lag is six hours and "real time" is a lie. You did not absorb a spike; you deferred OOM.

Layer 2's message queues exist to smooth spikes. They still need a bound and a policy when the bound is hit. Interviews: if you draw SQS/Kafka, they will ask what happens when consumers die. "The queue grows" needs a next sentence.

How it works

Three honest policies when the buffer is full:

Block the producer. TCP window / in-process queue. Can deadlock if the producer holds a lock the consumer needs.
Reject. 503/429, nack, QUEUE_FULL. Best for request/response. Pair with timeouts so threads do not sit on enqueue.
Drop. Logs, metrics, optional recs — never payments. Metric it.

Kafka/SQS pull is backpressure. Unbounded go func / fire-and-forget HTTP is not, until you add a bound.

End-to-end: a slow Elasticsearch should stall the indexer, which should stall (or reject) the ingest API, which should 503 the client. If any hop is unbounded, pressure stops there and RAM grows.

Load shedding is related: drop low priority when overloaded (health still 200, search 503). That is backpressure with a priority policy.

A simple example

Image upload: API writes to S3, enqueues resize on Redis list, returns 202.

Workers are down for 20 minutes. The list hits 50 million keys. Redis RAM dies. Uploads fail anyway, plus everything else on that Redis (sessions, cache).

Fix: bounded queue (max length). When full, API returns 503 "try later" or spills to SQS with its own limits and alarms on depth/age. Uploads wait or fail early. Session Redis stays up.

Kafka version: consumer lag alarm, and producers slow down or reject when lag exceeds a SLO — not when the disk is already full.

Common mistakes

Unbounded go func / thread-per-request / in-memory queue. The language makes it easy.

Calling a queue "backpressure" because it exists. A queue hides pressure until it is full. Bound it and alert on depth and age.

Dropping payments. Reject or block; do not silent-drop money.

Only limiting at the edge. Internal producers (cron, other services) bypass the API limit and still flood the queue.

No timeout on blocking backpressure. You converted "queue full" into "all threads stuck on enqueue."

How this shows up in real systems

TCP / HTTP/2 flow control, Reactive Streams, Akka, Node highWaterMark: explicit pull/window.
Kafka: consumers pull; lag is the signal. Producer max.block.ms when the buffer fills.
SQS: doesn't block producers; you alarm on queue depth and stop sending (or scale consumers).
Envoy / nginx overload and 503; Java thread pools reject with CallerRuns or abort.
Postgres will not "backpressure" your connection storm — it will hit max_connections. You put PgBouncer in front.

Recap

Backpressure makes producers slow down or fail when consumers cannot keep up.
Bound every queue; choose reject, block, or (rarely) drop.
Rate limit is the published cap; backpressure is today's fullness. Next: surviving machine loss with replication.