Vertical vs Horizontal Scaling

7 min read

In a Nutshell

When your system can't keep up with load, you have two fundamental moves. Vertical scaling (scale up) means making one machine more powerful — more CPU, more RAM, faster disks. Horizontal scaling (scale out) means adding more machines and spreading the work across them. Scaling up is simple but hits a hard ceiling (the biggest server money can buy) and gives you a single point of failure. Scaling out is theoretically unlimited and fault-tolerant, but forces you to solve hard distributed-systems problems: load balancing, data partitioning, and keeping state consistent across nodes. Almost every large system is a mix — scale up until it's painful, then scale out.

2D minimalistic diagram split in two halves: left half labeled "Scale Up" shows one server box growing taller and bulkier with bigger CPU/RAM icons; right half labeled "Scale Out" shows one server multiplying into a row of four identical smaller server boxes with a load balancer on top distributing arrows to each

How It Actually Works

The Two Axes of Scaling

Dimension	Vertical (Scale Up)	Horizontal (Scale Out)
What changes	One node gets bigger	More nodes get added
Upper limit	Hardware ceiling (largest available instance)	Practically unbounded
Fault tolerance	Single point of failure	Survives individual node loss
Complexity	Low — no app changes	High — needs LB, partitioning, coordination
Cost curve	Superlinear (top-tier hardware is disproportionately expensive)	Roughly linear (commodity hardware)
Downtime to scale	Often requires a reboot/migration	Add nodes with zero downtime
Data consistency	Trivial (single machine)	Hard (distributed state)

Why Vertical Scaling Hits a Wall

Scaling up is the path of least resistance: resize the VM, restart, done. No code changes, no distributed-systems headaches. But the ceiling is real. As of the mid-2020s, the largest cloud instances top out around 128–224 vCPUs and 24+ TB of RAM — and the price per unit of compute climbs steeply at the top end. Worse, a single big box means a single point of failure: when it goes down, everything goes down. There's also a maintenance problem — you can't patch or reboot without downtime unless you already have redundancy, which itself requires more than one machine.

Why Horizontal Scaling Is Harder

Adding machines sounds easy until you ask: how does a request find the right machine, and how do the machines agree on state? Scaling out forces you to confront:

Load distribution — you need a load balancer to spread requests (see Load Balancing).
State management — if a server holds session state in memory, the next request might hit a different server. This pushes you toward stateless services with shared state in Redis or a database.
Data partitioning — one database can't hold everything, so you shard (see Sharding & Partitioning).
Coordination — nodes must agree on who's the leader, who owns which data, and what the current config is (see Consensus & Leader Election).

Stateless Is the Enabler

The single most important design decision for horizontal scaling is making your application tier stateless. A stateless service keeps no client-specific data between requests — any node can handle any request. Session data, uploaded files, and caches all move to shared backing stores.

Stateful (hard to scale out):
  Client → Server A (holds session in local memory)
  Next request MUST return to Server A ("sticky sessions")

Stateless (easy to scale out):
  Client → [Any Server] → reads session from Redis
  Any node can serve any request → add nodes freely

Amdahl's & Universal Scalability Law

Scaling out never gives perfect linear returns. Amdahl's Law says the serial (non-parallelizable) fraction of your workload caps your maximum speedup. The Universal Scalability Law adds a second penalty: coordination and contention between nodes eventually make adding nodes hurt throughput. This is why doubling servers rarely doubles capacity — and why reducing shared state and cross-node chatter matters so much.

$2D minimalistic line graph with "Number of Nodes" on the x-axis and "Throughput" on the y-axis, showing three curves: a straight dashed line labeled "Ideal linear", a curve that flattens labeled "Amdahl (serial fraction)", and a curve that rises then dips down labeled "Universal Scalability Law (contention + coherency)"$

Seeing It in Action

Scenario: A startup's API is at 80% CPU during peak hours.

Stage 1 — Scale up (buy time):
  Resize instance:  4 vCPU / 16 GB  →  16 vCPU / 64 GB
  Effort: minutes. Buys ~4x headroom. No code changes.
  ✅ Good first move while you re-architect.

Stage 2 — Make the app stateless:
  Move sessions from in-process memory → Redis
  Move uploaded files from local disk → S3/object storage
  Now any instance can serve any request.

Stage 3 — Scale out:
  Put 6 instances behind a load balancer.
  Autoscale between 3 and 20 based on CPU.
  ✅ Fault tolerant, near-linear cost, no ceiling.

Stage 4 — Scale the data tier:
  Add read replicas (see replication-and-read-replicas.md)
  Shard when a single primary can't hold writes.

Rule of thumb: Scale up first because it's cheap in engineering effort; invest in scaling out before you hit the vertical ceiling, because that migration takes time you won't have during an outage.

Interview Questions

Q: What's the difference between vertical and horizontal scaling, and when would you choose each? Hint: Vertical = bigger machine (simple, has a ceiling, single point of failure). Horizontal = more machines (unbounded, fault-tolerant, complex). Choose vertical for quick wins, simple stateful systems (e.g., a single relational primary), or when you haven't hit the ceiling. Choose horizontal when you need fault tolerance, elasticity, or capacity beyond one machine.
Q: Why is making a service stateless so important for horizontal scaling? Hint: If a server holds client state (sessions, in-memory caches) locally, requests must be routed back to the same server (sticky sessions), which defeats even load distribution and breaks when that server dies. Stateless services push state to shared stores (Redis, DB, S3), so any node can serve any request — enabling free addition/removal of nodes.
Q: Doubling the number of servers rarely doubles throughput. Why? Hint: Amdahl's Law (the serial fraction of work can't be parallelized and caps speedup) and the Universal Scalability Law (coordination/contention overhead between nodes grows and can even reduce throughput past a point). Shared resources (a single database, locks) become bottlenecks. Reducing shared state and cross-node coordination improves scaling efficiency.
Q: What are the operational downsides of relying purely on vertical scaling? Hint: Hard hardware ceiling; superlinear cost at the top end; single point of failure; scaling and patching often require downtime (reboot/migration); no elasticity for spiky traffic. You also can't easily do rolling deployments without a second machine.
Q: You have a legacy monolith that stores sessions in local memory. Traffic is spiking. What's your migration path to horizontal scaling? Hint: First scale up to buy time. Then externalize state: move sessions to Redis, files to object storage, and make the app tier stateless. Add a load balancer, run multiple instances, then introduce autoscaling. Scale the data tier separately (read replicas, then sharding).

References

Designing Data-Intensive Applications by Martin Kleppmann — Chapter 1: Scalability
The Art of Scalability by Abbott & Fisher — the "AKF Scale Cube"
AWS: Scaling vertically vs horizontally — cloud-specific guidance

Dive Deeper

Universal Scalability Law — Neil Gunther — the math behind why scaling out has diminishing (then negative) returns
Google SRE Book — Handling Overload — what happens when scaling can't keep up
Systems Performance by Brendan Gregg — deep treatment of resource ceilings and capacity