Exponential Backoff

Pradeep P
3 days ago
3 min read

Series: Modern System Design · Layer 3 — Reliability

Layer 3 · Post 3 of 14

Layer 3 — Reliability · Post 28 of 88

Exponential backoff waits longer after each failed retry so a struggling service gets room to recover instead of being hammered.

What you'll learn

Why immediate retries keep a sick service sick
How exponential delays plus a cap actually look in numbers
Why jitter is not optional if thousands of clients retry together

The idea in one minute

You already decided to retry. Exponential backoff is the schedule: wait longer after each failure.

try 1 fail wait ~100 ms try 2 fail wait ~200 ms try 3 fail wait ~400 ms ... cap at e.g. 30 s, then DLQ or give up

The sick service gets gaps. Your client still recovers from blips (short wait) without becoming a denial-of-service bot (constant hammer).

Jitter means you add randomness so everyone does not wake up on the same millisecond.

Why it matters

Retries without delay are a thundering herd. Payments fails for 2 seconds. Every checkout in flight retries immediately. Payments comes back and is instantly crushed by the backlog of retries plus new traffic. It falls over again. You turned a blip into a sawtooth outage.

This is why AWS's architecture blogs treat backoff-plus-jitter as a default, not a nicety. Interviewers expect the phrase. They also expect you to cap it: unbounded exponential wait is how a webhook sits in a loop for hours on a 400 you should have dropped.

How it works

A common formula:

delay = min(cap, base * 2^attempt)
then delay = delay * random(0.5, 1.5) (or "full jitter": random(0, delay))

Example with base = 100 ms, cap 2 s:

After 1st fail: ~100 ms
After 2nd: ~200 ms
After 3rd: ~400 ms
After 4th: ~800 ms
After 5th: ~1.6 s
After 6th: cap at 2 s

On the user-facing request path, you rarely go past 2–3 attempts. The exponential curve is mostly for workers: SQS, webhook deliveries, connecting to a database at process start.

Respect Retry-After on 429. Their number beats your formula. That is rate limiting talking to you (a few posts from now).

Backoff is not a substitute for a circuit breaker. If the dependency is hard-down, waiting 100, 200, 400 ms still spends your timeout budget. Breakers fail fast without those tries. Use both: breakers to stop calling, backoff for the calls you still make.

A simple example

You send Stripe webhooks (Post 25). Their URL returns 500.

Immediate retry: you DDoS a customer who is deploying.
Exponential backoff over minutes, then hours, with a cap and a give-up: they get room to finish the deploy; you eventually dead-letter and page a human.

Same pattern for "cannot reach Postgres at boot." Kubernetes CrashLoopBackOff is exponential backoff for processes. Without it, a bad config restart-storms the node.

Common mistakes

No jitter. 10,000 lambdas time out together, sleep 1 s, retry together. You built a metronome.

No cap. 2^n becomes hours, then days, while a poison message is still wrong.

Backoff on the request path that exceeds the user timeout. The user already got 504. Your extra retries run for a ghost.

Linear retry ("every 1 s") for a dependency that is down for a minute. You still hit it 60 times per client. Exponential is the default; linear is for when you want a steady probe (health checks), not for user retries.

Treating backoff as fairness. It slows you. It does not magically shed load unless clients also stop sending new work (backpressure, rate limits).

How this shows up in real systems

AWS SDK, Google Cloud client libraries, Azure SDKs: exponential backoff with jitter is the default retry policy.
Kubernetes: CrashLoopBackOff, and controllers requeue with backoff.
Twilio / Stripe / GitHub webhook deliveries: retry windows measured in hours, not milliseconds.
Envoy retry backoff and gRPC retry policy (initialBackoff, maxBackoff, backoffMultiplier).

Recap

After each failed retry, wait longer, up to a cap.
Add jitter or synchronized clients become a stampede.
Request-path retries stay few and short; workers can back off for a long time — then you need idempotency.