Compute & Services
Where Your Code Actually Runs
Every system needs somewhere to execute its logic and some way to organize that logic into deployable, communicating pieces. This topic is about compute — the servers, containers, and functions that run your code — and services — how you structure, connect, discover, and secure the components that make up an application. The decisions here shape how you deploy, scale, and operate everything else: whether you run one monolith or fifty microservices, whether you package with containers or hand functions to a serverless platform, and how all those pieces find and talk to each other reliably.
What makes this topic cohesive is a single arc: as you decompose a system into more, smaller pieces for scalability and team autonomy, you gain flexibility but inherit a cascade of new problems — routing, discovery, resilience, security, and observability between the pieces. Much of modern infrastructure (API gateways, orchestration, service discovery, service meshes) exists precisely to tame the complexity that decomposition creates. Understanding this arc — the benefits of splitting and the machinery required to manage the split — is what lets you make the compute and services decisions deliberately rather than by default or fashion.
When This Comes Up
- System design interviews: "Monolith or microservices?" is a direct and frequent question, and the right answer (it depends on scale and team pressures) signals maturity. Beyond that, interviewers expect you to know where an API gateway sits, how services discover each other, when serverless fits, and how containers/orchestration underpin deployment. These are the operational realities behind every box you draw.
- Real architecture: Choosing a compute model (servers, containers, serverless), an application topology (monolith vs services), and the connective infrastructure (gateway, discovery, mesh) are foundational, hard-to-reverse decisions that determine deployment velocity, operational cost, and how the organization scales.
- Production operations: Rolling deployments, autoscaling, self-healing, service-to-service reliability, and zero-trust security all live here. The concepts in this topic are what keep a distributed system running, recoverable, and secure day to day.
How the Sub-Topics Connect
The sub-topics move from the anatomy of the request-serving tier (web vs app server) → the front door for many services (API gateway) → the central structural choice (monolith vs microservices) → the compute models that run the pieces (serverless, containers & orchestration) → and finally the connective tissue that lets many services operate reliably and securely (service discovery, service mesh):
1. Web Server & App Server
The two distinct servers a request passes through. A web server (NGINX, Apache) handles raw HTTP — TLS, static files, compression, reverse proxying — and is I/O-optimized and language-agnostic. An application server (Gunicorn, Tomcat, Node) runs your business logic — routing, DB queries, computation. Separating them buys performance (static content never touches your app), security (a hardened buffer), and independent scaling. The app server's concurrency model (process, thread, or async event-loop) and worker count determine how many requests run in parallel.
2. API Gateway
The single front door to a backend of many services, handling the cross-cutting concerns every API needs — authentication, rate limiting, routing, transformation, and aggregation (fulfilling one client request by fanning out to several services). It keeps individual services simple (they don't each reimplement auth and throttling) and hides internal topology from clients. It's the most application-aware member of the load-balancer/reverse-proxy/gateway spectrum, and the Backend-for-Frontend variant tailors a gateway per client type. The trade-off: it's a critical chokepoint that must be run as a resilient, scaled-out fleet.
3. Monolith vs Microservices
The central structural decision — and the most over-argued. A monolith is one deployable unit: simple, fast in-process calls, easy ACID transactions. Microservices split into independently-deployable services: independent scaling, team autonomy, and fault isolation, but enormous distributed-systems complexity. The modern consensus is monolith-first: start with a well-structured modular monolith and extract services only under real pressure (a component needs independent scaling, a team needs autonomy, a part needs isolated failure). The worst outcome is a "distributed monolith" — all the cost of microservices with none of the independence.
4. Serverless
Running code without managing servers: you write functions, the provider scales them from zero to thousands and back, and you pay only for execution time. It shines for event-driven, spiky, short, stateless workloads (image processing, webhooks, automation) where idle servers would waste money. Its defining constraint is the cold start — spinning up a fresh environment adds latency — alongside time limits, statelessness, and vendor lock-in. It's a poor fit for steady, long-running, latency-critical, or stateful workloads. "Serverless" also extends beyond functions to managed databases, queues, and serverless containers (Fargate, Cloud Run).
5. Containers & Orchestration
A container packages an app with its exact runtime and dependencies into a portable, immutable unit that runs identically everywhere — solving environment drift by sharing the host kernel (far lighter than VMs). Orchestration (overwhelmingly Kubernetes) manages many containers across a fleet: scheduling, self-healing, scaling, networking, and zero-downtime rolling updates. Kubernetes' heart is the declarative desired-state + reconciliation loop — you declare what you want ("5 healthy replicas") and the control loop continuously drives reality to match. This is the foundation of modern cloud-native deployment.
6. Service Discovery
The mechanism that answers "where is service X right now?" in a world where instances constantly come and go. A service registry tracks currently-healthy instances and their addresses through a register → health-check → discover → deregister lifecycle, so callers never route to a dead address. The key choice is client-side (the client queries the registry and picks an instance) vs server-side discovery (a load balancer/DNS name does it — as in Kubernetes Services). It's what decouples a service's stable identity from its ephemeral location, making autoscaling, redeploys, and self-healing possible without anyone editing addresses.
7. Service Mesh
The infrastructure layer that handles service-to-service communication, moving cross-cutting network concerns — mutual TLS, retries, timeouts, circuit breaking, load balancing, traffic routing, observability — out of application code and into sidecar proxies deployed beside each service. A control plane configures the sidecars (the data plane) declaratively, so capabilities like zero-trust mTLS and canary routing are added uniformly and language-agnostically, without touching app code. It complements the API gateway (mesh = internal east-west traffic; gateway = external north-south), and is worth its real operational complexity only once you have many services.
Sub-Topics
| # | Sub-Topic | What You'll Learn |
|---|---|---|
| 1 | Web Server & App Server | The two servers behind every request and how they divide labor |
| 2 | API Gateway | The single front door handling cross-cutting API concerns |
| 3 | Monolith vs Microservices | The central structural decision and when to split |
| 4 | Serverless | Running code without managing servers, and where it fits |
| 5 | Containers & Orchestration | Packaging and managing containers at scale with Kubernetes |
| 6 | Service Discovery | How services find each other in a dynamic environment |
| 7 | Service Mesh | Offloading service-to-service networking to infrastructure |