Load Balancer & Reverse Proxy

In a Nutshell

A reverse proxy is a server that sits in front of your application servers and forwards client requests to them, then relays the responses back — the client only ever talks to the proxy, never directly to the backends. A load balancer is a specialized reverse proxy whose primary job is distributing traffic across a pool of backends. The two concepts overlap heavily: most reverse proxies can load-balance, and most load balancers are reverse proxies. The distinction is one of emphasis — a reverse proxy is about what happens at the boundary (TLS, caching, routing, security), while a load balancer is about how traffic is spread across many servers.

How It Actually Works

Forward Proxy vs Reverse Proxy

These are mirror images — know the difference:

	Forward Proxy	Reverse Proxy
Sits in front of	Clients	Servers
Acts on behalf of	The client	The server
Client knows the real server?	Yes (proxy hides the client)	No (proxy hides the servers)
Typical use	Corporate egress, privacy, filtering	LB, TLS, caching, security, routing

Forward proxy:  [Clients] → (proxy) → Internet     (hides who's asking)
Reverse proxy:  Internet → (proxy) → [Servers]     (hides what's answering)

What a Reverse Proxy Does at the Boundary

Putting a reverse proxy in front of your app centralizes cross-cutting concerns so your backends don't each have to reimplement them:

Function	Benefit
Load balancing	Spread traffic across backends (see Load Balancing)
TLS termination	Decrypt once at the edge; backends speak plain HTTP internally
Caching	Serve cached responses without hitting backends
Compression	gzip/brotli responses centrally
Routing	Path/host-based routing (/api → service A, /img → service B)
Security	WAF, rate limiting, IP allow/deny, hide backend topology
Request buffering	Absorb slow clients so backends aren't tied up
Observability	One place for access logs, metrics, tracing headers

Load Balancer vs Reverse Proxy vs API Gateway

A frequent point of confusion. They form a spectrum of increasing application-awareness:

	Load Balancer	Reverse Proxy	API Gateway
Primary job	Distribute traffic	Boundary functions + LB	API-specific control plane
Awareness	L4 or L7	L7 (HTTP)	Deep app/API awareness
Adds	Health checks, distribution	TLS, cache, routing, security	Auth, rate limits, versioning, aggregation, transformation
Example	AWS NLB, HAProxy	NGINX, Envoy	Kong, AWS API Gateway

Rule of thumb: a load balancer spreads load, a reverse proxy also handles edge concerns, and an API gateway (see API Gateway) adds API management on top. In practice one component (e.g., Envoy or NGINX) can play all three roles.

TLS Termination vs Passthrough

Termination (common):
  Client ──HTTPS──▶ [Reverse Proxy decrypts] ──HTTP──▶ Backend
  ✅ Backends simpler, central cert management, can inspect/cache
  ⚠️ Internal hop is plaintext (mitigate with mTLS / private network)

Passthrough / re-encryption (high security):
  Client ──HTTPS──▶ [Proxy] ──HTTPS/mTLS──▶ Backend
  ✅ Encrypted end-to-end
  ⚠️ Proxy can't inspect/cache; more overhead

Why It Matters for System Design

The reverse proxy is where you enforce policy once: it's the single chokepoint for TLS, auth pre-checks, rate limiting, and routing — so backends stay simple and stateless. It also hides your topology (clients can't tell how many backends exist or where they are), which improves both security and operational flexibility (add/remove/relocate backends transparently).

Seeing It in Action

Scenario: NGINX as a reverse proxy doing TLS termination, routing, and load balancing.

# Upstream backend pools
upstream web_app  { server 10.0.1.10:8080; server 10.0.1.11:8080; }
upstream img_svc  { server 10.0.2.10:9090; server 10.0.2.11:9090; }

server {
    listen 443 ssl http2;
    server_name example.com;

    ssl_certificate     /etc/ssl/example.crt;   # TLS terminated here
    ssl_certificate_key /etc/ssl/example.key;

    gzip on;                                     # central compression

    # Path-based routing to different backend services
    location /images/ {
        proxy_pass http://img_svc;
        proxy_cache img_cache;                   # cache images at the proxy
        proxy_cache_valid 200 1h;
    }

    location / {
        proxy_pass http://web_app;               # everything else → web app
        proxy_set_header X-Forwarded-For $remote_addr;   # preserve client IP
        proxy_set_header X-Forwarded-Proto https;
        limit_req zone=perip burst=20 nodelay;   # rate limit per client IP
    }
}

What this single config centralizes: TLS is terminated once (backends speak plain HTTP), traffic is routed by path to the right service pool and load-balanced within each, images are cached at the edge, responses are compressed, the real client IP is forwarded, and per-IP rate limiting protects the backends — none of which the application servers have to implement themselves. This is the reverse proxy as the system's front door and policy enforcement point.

Interview Questions

1. Q: What's the difference between a forward proxy and a reverse proxy? Hint: A forward proxy sits in front of clients and acts on their behalf (corporate egress, privacy, filtering) — it hides the client. A reverse proxy sits in front of servers and acts on their behalf (LB, TLS, caching, security) — it hides the backends. Clients knowingly use a forward proxy; they're usually unaware of a reverse proxy.

2. Q: How do a load balancer, reverse proxy, and API gateway differ? Hint: Increasing app-awareness: a load balancer distributes traffic (L4/L7) with health checks; a reverse proxy adds edge functions (TLS termination, caching, routing, security) plus LB; an API gateway adds API-management (auth, rate limiting, versioning, request aggregation/transformation). They overlap heavily and one product can play all three roles.

3. Q: What is TLS termination, and what are the trade-offs vs passthrough? Hint: Termination decrypts HTTPS at the proxy, so backends speak plain HTTP — simpler backends, central cert management, and the proxy can inspect/cache/route. Downside: the internal hop is plaintext (mitigate with mTLS/private networking). Passthrough/re-encryption keeps traffic encrypted end-to-end (higher security) but the proxy can't inspect or cache and incurs more overhead.

4. Q: Why put a reverse proxy in front of your application servers at all? Hint: It centralizes cross-cutting concerns (TLS, caching, compression, routing, rate limiting, WAF, logging) so backends stay simple and stateless, provides a single policy-enforcement chokepoint, hides backend topology (security + flexibility to add/remove/relocate servers transparently), and buffers slow clients to protect backends.

5. Q: A reverse proxy terminates TLS and forwards to backends. How do backends still see the real client IP and protocol? Hint: The proxy injects forwarding headers — X-Forwarded-For (original client IP), X-Forwarded-Proto (original scheme, e.g., https), and X-Forwarded-Host. Backends read these instead of the proxy's own connection info. The proxy must be trusted and should strip/overwrite these headers from untrusted clients to prevent spoofing.

References

• NGINX: What is a reverse proxy? — definitions and functions
• Cloudflare: Reverse proxy vs forward proxy — the distinction clearly explained
• Envoy Proxy documentation — a modern L7 proxy powering many gateways/meshes

Dive Deeper

• HAProxy Configuration Manual — a battle-tested L4/L7 proxy in depth
• Google Maglev — network load balancing at scale
• The AWS Builders' Library: reverse proxies and dependency isolation — proxies in resilient architectures