Load Balancers
- Pradeep P
- 4 days ago
- 4 min read
Layer 1 · Post 4 of 15
← Previous: Horizontal vs Vertical Scaling → Next: Reverse Proxies
Layer 1 — The building blocks · Post 4 of 88
A load balancer sits in front of many servers and spreads traffic so no single machine becomes the bottleneck or the single point of failure.
What you'll learn
What a load balancer does besides "split traffic"
The common ways it picks a backend (round robin, least connections, hashing)
Why health checks and the load balancer's own failure domain matter
The idea in one minute
You scaled out. You now have several copies of your app. Clients should not have to know their names.
A load balancer (LB) is the front door:
Clients send requests to one address (the LB).
The LB forwards each request to one healthy backend.
If a backend dies, the LB stops sending it work.
Clients → [ Load balancer ] → App A
→ App B
→ App C
It turns many machines into one URL.
Why it matters
Without an LB, horizontal scaling is a rumor. You would hand users a list of IPs, or DNS-round-robin with no idea which server is dead.
With an LB you get:
Even-ish load so one hot server does not melt while others sit idle.
Zero-downtime deploys: drain a server, deploy, put it back.
Health-aware routing: do not send traffic to a process that is failing its checks.
The LB is also a new thing that can fail. Real designs run more than one load balancer, often with DNS or anycast in front of them.
How it works
Layer 4 vs Layer 7
You will hear these in interviews. They are OSI nicknames:
Layer 4 (transport): the LB looks at IP and port (TCP/UDP). It is fast. It does not read HTTP paths or headers. A TCP connection is typically pinned to one backend for its life.
Layer 7 (application): the LB understands HTTP. It can route /images to one pool and /api to another, add headers, terminate TLS, and retry idempotent GETs.
Most public websites use Layer 7 at the edge (or a CDN that behaves like one). Internal service meshes often mix both.
How it picks a backend
Strategy: Round robin; Idea: Take turns; Watch-out: Uneven if some requests are much heavier
Strategy: Least connections; Idea: Prefer the server with fewest open requests; Watch-out: Needs a good view of "busy"
Strategy: Weighted; Idea: Some servers get more traffic (bigger VMs); Watch-out: Weights go stale
Strategy: Consistent hashing; Idea: Same client or key tends to hit the same server; Watch-out: Useful for caches; bad if that server dies and takes hot keys with it
Health checks
The LB periodically asks each backend "are you ok?" — a cheap HTTP /health or a TCP connect.
If checks fail, the server is pulled from the pool. If you make /health too heavy (it hits the database), a database blip marks every app instance unhealthy and you take the whole site down yourself.
What it does not do by itself
A load balancer spreads requests. It does not magically scale a single database. All those app servers can still stampede one primary. The LB solved the stateless tier. Data is still later in this layer.
A simple example
Three identical API containers. The LB is round-robin.
Request 1 → API-1 Request 2 → API-2 Request 3 → API-3 Request 4 → API-1 …
API-2's disk fills and /health starts failing. The LB marks API-2 out. Requests now alternate between API-1 and API-3. Users do not get a list of errors from API-2. They might get a bit more latency because two servers share the load that three used to share.
You deploy a new version: take API-1 out of the pool, replace it, health-check it, put it back. Repeat. That is a rolling deploy. The LB is the mechanism.
Common mistakes
One load balancer, no plan if it dies. You moved the single point of failure up a layer. Use a pair, or a cloud LB that is already a fleet.
Health checks that test the universe. Health should mean "this process can accept work," not "the entire company is having a good day."
Sticky sessions as a default. Pinning a user to one server so in-memory session works fights the whole point of a pool. Store session state in Redis or a DB.
Forgetting slow backends. Round robin will still send a fair share to a server that is crawling. Least-connections or latency-aware routing handles that better.
How this shows up in real systems
AWS ALB/NLB, GCP Load Balancing, Azure LB, nginx, HAProxy, Envoy: the usual names. Cloud LBs are managed fleets. nginx/HAProxy are software you run yourself.
Kubernetes Service / Ingress: kube-proxy or a controller programs an LB so pods can come and go.
DNS + anycast: global systems (Cloudflare, Google) advertise the same IP from many locations; the "load balancer" is the edge of the internet.
In a design interview, drawing a load balancer in front of N app servers is the standard first box after "clients."
Recap
A load balancer hides many backends behind one address and skips the unhealthy ones.
Layer 4 is fast and simple; Layer 7 can route on HTTP and terminate TLS.
Pair it with boring health checks, and do not let it become the new single point of failure.
A close cousin sits in the same neighborhood: the reverse proxy. It often is the load balancer, with extra jobs.
Layer 1 · Post 4 of 15
← Previous: Horizontal vs Vertical Scaling → Next: Reverse Proxies
Comments