Service Discovery

7 min read

In a Nutshell

In a dynamic system where services run on many machines and instances constantly come and go — scaling up, crashing, redeploying, moving between hosts — how does one service find the current network address of another? Hard-coding IP addresses is hopeless when those addresses change every few minutes. Service discovery is the mechanism that answers "where is service X right now?" It maintains a live registry of healthy service instances and their locations, so callers always route to instances that actually exist and are healthy. It's the invisible plumbing that makes microservices, autoscaling, and container orchestration work.

2D minimalistic diagram showing a service registry box in the center holding a live list of service instances and their addresses; service instances register themselves on startup (arrows in) and are health-checked; a client service queries the registry to discover a healthy instance address and then connects to it, illustrating dynamic address lookup

How It Actually Works

The Problem: Ephemeral Addresses

Static world (old):  service B always at 10.0.0.5 → hard-code it. Fine.

Dynamic world (now):
  Service B runs as 8 autoscaled containers, addresses change constantly:
    10.0.3.11, 10.0.7.42, 10.0.2.9, ...  (and a new set after each deploy)
  Instances crash, scale, and move every few minutes.
  Hard-coded addresses break immediately. 💥
  → You need a live directory that always knows where B's healthy
    instances are RIGHT NOW.

The Service Registry

At the heart of service discovery is a service registry — a database of currently-available service instances. Its lifecycle:

1. REGISTER   instance starts → registers (name, address, port) in the registry
2. HEALTH     registry health-checks instances (or they send heartbeats)
3. DISCOVER   a caller queries "where is service B?" → gets healthy instances
4. DEREGISTER instance stops/fails/misses heartbeats → removed from registry

Only healthy, live instances stay in the registry, so callers never get a dead address.

Client-Side vs Server-Side Discovery

The key architectural choice is who queries the registry:

	Client-Side Discovery	Server-Side Discovery
Who queries the registry	The client itself	A load balancer / router
Client picks instance?	Yes (client-side LB)	No (LB does it)
Client complexity	Higher (registry-aware)	Lower (just call a stable endpoint)
Examples	Netflix Eureka + Ribbon	K8s Services, AWS ELB
Extra hop?	No	Yes (through the LB)

Client-side:  Client → asks Registry → gets [instances] → picks one → calls it
Server-side:  Client → calls a stable LB/DNS name → LB queries Registry → routes

Server-side (used by Kubernetes) keeps clients simple — they call a stable name and the platform handles discovery and load balancing. Client-side gives clients control at the cost of embedding discovery logic in every service.

Registration: Self vs Third-Party

Self-registration — the instance registers/deregisters itself and sends heartbeats. Simple but couples the app to the registry.
Third-party registration — a separate registrar watches for instances (e.g., via the orchestrator) and registers them. The app stays unaware. This is what Kubernetes does — the platform registers pods automatically.

Service Discovery in Kubernetes

Kubernetes bakes service discovery in, so you rarely run a separate registry:

A "Service" object gives a stable virtual IP + DNS name (e.g., "orders").
  → Any pod can reach the orders service at http://orders
  → K8s tracks which pods back that Service (via labels) and load-balances
  → Pods come and go; the Service name stays stable; discovery is automatic
  → kube-proxy / DNS (CoreDNS) handle the routing under the hood

This is server-side discovery via DNS: services find each other by name, and the platform resolves names to healthy instances.

Common Tools

Tool	Role
Kubernetes Services + CoreDNS	Built-in DNS-based discovery
Consul	Registry + health checks + DNS/HTTP interface
etcd / ZooKeeper	Consistent registry backing store (see Consensus)
Netflix Eureka	Client-side discovery registry
AWS Cloud Map / ELB	Managed discovery + routing

2D minimalistic diagram comparing client-side and server-side discovery: top row shows a client querying the registry directly, receiving instance addresses, and calling one; bottom row shows a client calling a stable load-balancer endpoint that consults the registry and routes to a healthy instance, with the client unaware of the registry

Seeing It in Action

Scenario: An order service calling an inventory service in Kubernetes.

Without service discovery (broken):
  order-service hard-codes inventory at 10.0.4.7:8080
  → inventory autoscales / redeploys → new pod IPs → order-service
    calls a dead address → errors. Every scale event breaks it. 💥

With Kubernetes service discovery (works):
  1. inventory Deployment runs 6 pods with label app=inventory.
  2. An "inventory" Service selects those pods and gets a stable
     DNS name + virtual IP.
  3. order-service simply calls:  http://inventory:8080/stock/123
  4. CoreDNS resolves "inventory" → the Service's virtual IP.
  5. kube-proxy load-balances the request to a HEALTHY inventory pod.
  6. When inventory scales to 12 pods, or a pod crashes and is replaced,
     the Service automatically tracks the current healthy set.
     order-service's code NEVER changes — it always calls "inventory".

Health integration:
  - A failing inventory pod fails its readiness probe → removed from the
    Service's endpoints → no traffic routed to it → self-healing discovery.

Why this is foundational: service discovery is what decouples identity (the logical service "inventory") from location (whichever pods happen to be running it right now). Callers reference the stable name; the platform continuously maps that name to the live, healthy instances behind it. This is precisely what lets you autoscale, redeploy, and recover from failures without anyone updating addresses — and it's why microservices and container orchestration are viable at all. Without it, every scaling event or deploy would break inter-service calls.

Interview Questions

Q: What problem does service discovery solve? Hint: In dynamic systems, service instances constantly change addresses (autoscaling, crashes, redeploys, moving hosts), so hard-coded IPs break immediately. Service discovery maintains a live registry of healthy instances and their locations, letting callers always find and route to instances that currently exist and are healthy. It decouples a service's logical identity from its ephemeral network location.
Q: How does a service registry work through its lifecycle? Hint: Register (instances register their name/address/port on startup), health-check (registry probes or instances heartbeat), discover (callers query "where is service X?" and get healthy instances), and deregister (failed/stopped instances that miss heartbeats are removed). This keeps the registry reflecting only live, healthy instances so callers never receive a dead address.
Q: Compare client-side and server-side service discovery. Hint: Client-side: the client queries the registry, gets the instance list, and picks one itself (client-side load balancing) — more client complexity, no extra hop (Eureka/Ribbon). Server-side: the client calls a stable endpoint (LB/DNS name) and a load balancer/router queries the registry and routes — simpler clients, one extra hop (Kubernetes Services, ELB). K8s uses server-side via DNS.
Q: How does Kubernetes provide service discovery? Hint: Via Service objects and DNS. A Service gives a stable virtual IP and DNS name; it selects backing pods by label and load-balances across them. Callers reach the service by name (e.g., http://inventory), CoreDNS resolves it, and kube-proxy routes to a healthy pod. Pods register/deregister automatically (third-party registration by the platform), and unhealthy pods are removed via readiness probes.
Q: What's the difference between self-registration and third-party registration? Hint: Self-registration: each instance registers/deregisters itself and sends heartbeats — simple but couples the app code to the registry. Third-party registration: a separate registrar (often the orchestrator) detects instances and registers them, so the app stays unaware — this is what Kubernetes does by automatically tracking pods. Third-party keeps services decoupled from discovery infrastructure.

References

Microservices.io: Service Discovery patterns — client-side, server-side, registry
Kubernetes: Service and DNS — built-in discovery
HashiCorp Consul — registry + health + DNS

Dive Deeper

Netflix Eureka — client-side discovery at scale
etcd and the Raft-backed registry — the consistent store under Kubernetes discovery
Building Microservices by Sam Newman — discovery in microservice architecture