Horizontal vs Vertical Scaling
- Pradeep P
- 3 days ago
- 4 min read
Layer 1 · Post 3 of 15
← Previous: Requests, Responses and Latency → Next: Load Balancers
Vertical scaling makes one machine bigger. Horizontal scaling adds more machines. Most internet systems eventually have to do the second.
What you'll learn
What "scale up" and "scale out" actually change
Why vertical scaling is simpler — and why it stops working
What has to be true of your app before you can add machines
The idea in one minute
Traffic grew. The system is slow or falling over. You have two moves:
Vertical scaling (scale up): keep one machine, give it more CPU, RAM, or a faster disk.
Horizontal scaling (scale out): keep the same size of machine, run more copies of it.
Vertical is a bigger elephant. Horizontal is more elephants. The internet is made of more elephants.
Why it matters
Scaling is the plot of almost every system design problem. "It works for 1,000 users" is not a design. The design is what you do at 10× and 100×.
Vertical scaling is the right first move more often than blog posts admit. It is also a dead end. There is a largest VM the cloud will sell you. There is a largest disk a database can chew. And one machine is still one failure domain — when it dies, everything dies.
Horizontal scaling is how you get past that wall. It is also how you inherit every problem from post 1: more machines, more network, more partial failure.
How it works
Vertical
You run one API process. It maxes out CPU. You change the instance from 4 cores / 16 GB to 16 cores / 64 GB. Throughput goes up. You did not change the code.
Limits:
Price grows faster than capacity at the top end.
One machine still has one network card, one rack, one datacenter.
Some workloads do not get faster with more cores (one giant lock, one disk, one thread).
Horizontal
You run N copies of the API behind something that spreads traffic (the next post: load balancers). Each copy handles a share of requests.
This only works if copies are interchangeable. That usually means:
No important state on the local disk. Session data lives in a database or Redis, not in one server's memory.
Requests can land on any copy. If "user 17 must always hit server B," you do not really have N servers. You have a routing mess.
Work can be split. Stateless request handlers scale out easily. A single-writer database does not.
So teams scale stateless tiers horizontally first (web, API, workers), and treat stateful tiers (databases) more carefully — replication and sharding come later in this layer.
A mixed reality
Real systems do both. You pick a sensible machine size (vertical), then add copies (horizontal), then maybe shard the database when even a big primary cannot keep up.
Autoscaling (Layer 5) is horizontal scaling on a timer: add copies when CPU or request queue grows, remove them when it shrinks.
A simple example
A to-do API stores tasks in PostgreSQL.
1,000 users: one small VM for the API, one small VM for Postgres. Fine.
50,000 users: the API CPU is hot. You scale the API vertically once. Then you scale it horizontally to three copies. Postgres is still one machine, because three APIs can share one database for a long time.
5 million users: Postgres CPU and disk are the bottleneck. A bigger Postgres (vertical) buys time. Eventually you read replicas or shard. The API farm is already 50 copies.
The lesson: scale the part that is actually tight, and know which parts are easy to copy.
Common mistakes
Scaling the wrong layer. Ten more app servers will not help if every request waits on one overloaded database.
Sticky in-memory sessions. You added three APIs, then pinned each user to one of them so their shopping cart survived. You have re-created one machine, with extra complexity.
Equating "Kubernetes" with "scaled." Orchestration makes copies easier to run. It does not make a stateful process safe to copy.
Ignoring cost. Horizontal scaling can be cheaper until you over-provision and pay for idle copies 24/7.
How this shows up in real systems
Web/API fleets at Google, Shopify, or a typical SaaS company: dozens to thousands of identical stateless pods.
Databases: vertical as far as it is sane, then replicas, then shards.
Batch jobs: more workers pulling from a queue. The queue is what makes the copies interchangeable.
Interview cue: if you only say "we'll add more servers," ask yourself which servers, and whether they hold state.
Recap
Vertical: bigger machine. Simple, limited, still one failure.
Horizontal: more machines. How large systems grow — if the work can be split and state lives elsewhere.
You almost always scale stateless compute out first, and treat data as the hard part.
Once you have more than one API server, something has to sit in front and spread the traffic. That something is a load balancer.
Layer 1 · Post 3 of 15
← Previous: Requests, Responses and Latency → Next: Load Balancers
Comments