Sharding & Partitioning

7 min read

In a Nutshell

Replication and caching scale reads, but eventually your write volume or dataset size outgrows what a single machine can hold. Sharding is the answer: split the data horizontally across many independent database instances, each owning a slice of the rows. A shard key decides which shard a given row lives on, and a routing layer sends each query to the right place. This is the scalability lens on partitioning — where Topic 04 focuses on the mechanics of shard keys and strategies, here we focus on sharding as a scaling technique: what it unlocks, what it costs, and how to sequence it against your other scaling tools.

This topic complements Topic 04 — Sharding & Partitioning, which covers shard-key selection and consistent hashing in depth. Read that for the internals; read this for the scaling strategy.

2D minimalistic diagram showing one overloaded database box on the left labeled "write-bound: too many writes for one node", an arrow labeled "shard", and three database boxes on the right each handling one-third of the writes, with a router directing incoming writes to the correct shard

How It Actually Works

Why Sharding Is the "Last Resort" of Scaling

Sharding is powerful but expensive in complexity, so it belongs at the end of your scaling playbook, not the start:

Scaling ladder for the data tier (climb in order):
  1. Query & schema optimization  (indexes, better queries)
  2. Vertical scaling             (bigger DB instance)
  3. Caching                      (offload reads to Redis)
  4. Read replicas                (scale reads horizontally)
  5. Sharding                     (scale WRITES + dataset size)  ← only when 1–4 exhausted

Replicas and caches only scale reads — every write still hits the single primary, and the whole dataset must still fit on one machine. Sharding is the only technique that scales write throughput and total storage horizontally. It's also the hardest to undo.

What Sharding Unlocks vs What It Breaks

Unlocks	Breaks
Horizontal write throughput	Cross-shard joins (must assemble in app code)
Storage beyond one machine	Cross-shard transactions (need 2PC/sagas)
Smaller per-shard indexes (faster)	Global aggregations (scatter-gather)
Blast-radius isolation per shard	Auto-increment IDs (collisions across shards)
Independent per-shard scaling	Rebalancing and resharding (operationally painful)

Partitioning Strategies at a Glance

Strategy	Routing	Strength	Weakness
Range	Key falls in a range	Efficient range scans	Hotspots on sequential keys
Hash	`hash(key) % N`	Even distribution	Range queries hit all shards
Consistent Hash	Key placed on a ring	Minimal movement when resizing	More complex to operate
Directory	Lookup table maps key→shard	Maximum flexibility, easy rebalancing	Extra hop; directory must be HA
Geo / entity	By region or tenant	Data locality, compliance	Uneven tenant sizes

The Routing Layer

Something must translate a query into "which shard(s)?" Three common places to put that logic:

Client-side — the app library knows the shard map (fast, but every client must stay in sync).
Proxy/router tier — a dedicated layer (e.g., Vitess vtgate, ProxySQL) hides sharding from the app.
Coordinator inside the DB — the database itself routes (e.g., Citus, CockroachDB, MongoDB mongos).

        ┌─────────┐
Client ─▶  Router  │──▶ Shard Map (key ranges → shard)
        └────┬────┘
      ┌──────┼──────┐
      ▼      ▼      ▼
   Shard1  Shard2  Shard3   (each an independent primary + its own replicas)

Note that each shard is usually itself replicated — so a real system combines sharding (for write scale) with replication (for read scale and availability).

Rebalancing Without Downtime

The hardest operational problem is moving data when a shard gets hot or full. Techniques:

Fixed partition count — pre-create many more logical partitions than nodes (e.g., 1024), then move whole partitions between nodes. Used by Elasticsearch, Riak.
Consistent hashing with virtual nodes — each physical node owns many ring positions; adding a node steals a fraction from each existing node.
Split/merge — dynamically split a range partition when it grows too large (HBase, CockroachDB).

Avoid hash % N sharding precisely because changing N reshuffles nearly everything.

2D minimalistic diagram showing a hot shard (shard 2) larger and redder than the others, with a "split" arrow dividing it into two balanced shards, and a note showing that only shard 2's data moves while shards 1 and 3 stay untouched

Seeing It in Action

Scenario: Scaling a multi-tenant SaaS analytics platform.

Problem: single Postgres primary at 90% write capacity; 40 TB and growing.
Reads already offloaded to 3 replicas + Redis. Writes are the bottleneck.

Decision: shard by tenant_id (directory-based routing)

Why tenant_id:
  ✅ Each tenant's data + queries stay on one shard (no cross-shard joins
     for the dominant query pattern: "dashboard for tenant X")
  ✅ Natural isolation — a noisy tenant can't slow others' shards
  ✅ Compliance: EU tenants can be pinned to EU shards
  ✅ Directory routing lets us MOVE a big tenant to a dedicated shard

Handling the giant tenant (10x others):
  → Give the whale its own dedicated shard(s)
  → Small tenants co-located many-per-shard
  → Directory table makes this a metadata change, not a rehash

Cross-tenant analytics (rare):
  → Not served from OLTP shards. Stream all shards → data warehouse
    (see 19-advanced-topics/data-pipelines-etl-elt.md) for global reporting.

Key insight: the shard key was chosen to make the common query single-shard and to keep the painful operational task (moving a large tenant) a cheap metadata update rather than a full rehash.

Interview Questions

Q: Why is sharding usually the last scaling technique you reach for, not the first? Hint: It's the only technique that scales writes and total storage horizontally, but it introduces cross-shard joins/transactions, global-aggregation pain, ID-generation issues, and hard rebalancing. Cheaper options (indexing, vertical scaling, caching, read replicas) should be exhausted first because they scale reads without that complexity.
Q: How do sharding and replication work together? Hint: They solve different problems. Sharding splits the dataset to scale writes and storage; replication copies each shard to scale reads and provide availability. Production systems combine them: each shard is an independent primary with its own replicas.
Q: What are the ways to implement the routing layer, and their trade-offs? Hint: Client-side (fast, but every client must know the shard map and stay in sync), proxy/router tier (transparent to apps, one more hop and component to run — Vitess, ProxySQL), or database-native coordinator (Citus, Mongo mongos, CockroachDB). Trade-off is transparency vs. added latency/operational surface.
Q: How do you rebalance a sharded system without a full data reshuffle? Hint: Avoid hash % N. Use a fixed large number of logical partitions moved wholesale between nodes, consistent hashing with virtual nodes (adding a node steals a slice from each), or dynamic split/merge of range partitions. Directory-based routing lets you move individual hot keys as a metadata change.
Q: You sharded by user_id, but one "user" is actually a huge enterprise account causing a hot shard. What now? Hint: This is data skew. Options: give the whale a dedicated shard (directory routing makes this cheap), sub-shard that entity by a secondary key, split the range, or introduce a compound key. Prevention: monitor per-shard size/traffic and choose keys with even distribution and no single dominant value.

References

Designing Data-Intensive Applications by Martin Kleppmann — Chapter 6: Partitioning
Vitess — YouTube's MySQL sharding/routing layer
Citus — distributed PostgreSQL that shards transparently

Dive Deeper

Instagram's Sharding IDs — generating globally-unique IDs across shards
Slack: Scaling Datastores by Cellular Architecture — sharding a large production system
Notion: Sharding Postgres — a real, recent migration to a sharded architecture