SLOs, SLIs, and Error Budgets
- Pradeep P
- 2 days ago
- 3 min read
Layer 5 · Post 15 of 15
← Previous: Cell-Based Architecture → Next: The End-to-End Argument
Layer 5 — Modern infrastructure · Post 114 of 119
An SLI is the measurement, an SLO is the target, and the error budget is how much unreliability you will spend on change.
What you'll learn
How SLI / SLO / SLA differ — and which one engineers own
Why 99.9% is a budget of minutes, not a vibe
How error budgets decide "ship vs freeze" without a personality fight
The idea in one minute
SLI (service level indicator): a number from production. Example: fraction of HTTP requests that complete in < 300 ms with 2xx/3xx/4xx (not 5xx) over 30 days. Be precise: which requests, which status codes, which percentile.
SLO (objective): the target for that SLI. Example: SLI ≥ 99.9%.
Error budget: 1 − SLO. At 99.9%, you may "spend" 0.1% failed-or-slow requests. Over 30 days that is a finite count of bad events. When the budget is gone, you slow down deploys and fix reliability — unless the product explicitly raises the SLO (costs money).
SLA: the contract with a customer (credits, lawyers). Often looser than the internal SLO so you do not pay refunds every time you miss a tight goal. Do not confuse them.
SLI: what we measure
SLO: what we aim at
budget: SLO minus reality (headroom for change and failure)
SLA: what we promise outsiders
Post 56 (observability) is how you see. This post is what "good" means.
Why it matters
Without an SLO, every outage is a moral argument. With one, a canary that burns 40% of the monthly budget in an hour is an automatic rollback conversation.
Interviews: they want user-centric SLIs (checkout success), not "CPU < 80%." CPU is a cause; the user felt latency or errors. CAP/PACELC choices should show up as different SLOs for different APIs, not one company-wide 99.99% sticker.
How it works
Pick the user journey. "Availability of the load balancer" can be green while checkout is dead (dependency). SLI on the action: PlaceOrder success.
Availability SLI. Good events / valid events. Exclude bot scrapes and client 404s you do not control — or you will never hit the SLO. Include your 500s and timeouts.
Latency SLI. Percentile over a window (p99 < 300 ms), or fraction of requests faster than a threshold (often easier to budget). Little's Law and Tail at Scale (later) explain why p99 moves when you fan out.
Window. Rolling 28/30 days is common. A 5-minute SLO is a pager; a 30-day SLO is a product target.
Error budget policy. Budget remaining → ship. Budget spent → feature freeze, reliability work, or raise the SLO (honest: more replicas, more $). Burning budget on toil vs launches is a management choice the SLO makes visible.
99.9% vs 99.99%. Extra nine is often 10× the engineering (multi-region, cells, paging). Do not "plus one nine" in a slide.
Multi-SLO. Availability and latency. A 100% "up" service that is 10 s slow fails the latency SLO.
A simple example
Checkout API, 30 days, 100 million valid requests, SLO 99.9% availability.
Budget = 100e6 × 0.001 = 100,000 bad requests. A bad deploy that 500s for 2 minutes at 5,000 RPS burns 600,000 — six months of budget. Rollback is not drama; it is arithmetic.
If the team wants weekly risky deploys, they need more budget (lower SLO) or safer deploys (canaries, cells, faster rollback). The SLO makes that trade explicit.
Common mistakes
SLO on CPU / disk. Users do not buy CPU. Use it as an alert, not the objective.
Counting 4xx as failures. User typos should not burn the budget (usually). Your 429s from a mis-set limiter might.
100% SLO. No budget ⇒ no deploys, or you lie. FLP and fallacies say 100% is a fantasy on a network.
SLA = SLO. Then finance owns the pager math. Keep an internal SLO tighter.
Too many SLIs. Three that people can name beat thirty dashboards (Post 56).
How this shows up in real systems
Google SRE book: the source of the vocabulary.
Cloud provider SLAs: credits at 99.9/99.95 — their contract, not your product SLO.
Datadog / Prometheus / SLO-as-code: burn-rate alerts (fast vs slow burn).
Error budget meetings: engineering and product, not only SRE.
Recap
SLI measure, SLO target, budget = permission to change, SLA lawyers.
User journeys and finite bad events beat "five nines" as a personality.
Next layer: named ideas that cut across stacks — the end-to-end argument.
Layer 5 · Post 15 of 15
← Previous: Cell-Based Architecture → Next: The End-to-End Argument
Comments