Circuit Breaker Pattern

In a Nutshell

The circuit breaker is a resilience design pattern that prevents an application from repeatedly trying an operation that's likely to fail — protecting it from a failing dependency and giving that dependency room to recover. Like an electrical breaker that trips to protect a circuit from a surge, the software circuit breaker "trips open" after too many failures and immediately rejects calls (failing fast) instead of hanging on a broken service. This topic covers it as an implementable pattern — its state machine, the code, and the design decisions — complementing the availability-focused treatment in Topic 06 — Circuit Breaker.

How It Actually Works

The Three-State Machine

The circuit breaker is a State pattern (see Structural & Behavioral Patterns) with three states:

        failures ≥ threshold
   CLOSED ──────────────────────▶ OPEN
     ▲                              │
     │ trial succeeds               │ cooldown timer elapses
     │                              ▼
     └────────────────────────  HALF-OPEN
              trial fails ──────────┘  (allows a few test calls)

CLOSED    → calls flow normally; failures are counted.
OPEN      → calls rejected instantly (fail fast) for a cooldown period.
HALF-OPEN → a few trial calls test recovery; success closes, failure re-opens.

Implementing It

class CircuitBreaker:
    def __init__(self, fail_max=5, reset_timeout=30):
        self.fail_max = fail_max
        self.reset_timeout = reset_timeout
        self.failures = 0
        self.state = "CLOSED"
        self.opened_at = None

    def call(self, fn, *args):
        if self.state == "OPEN":
            if time.time() - self.opened_at >= self.reset_timeout:
                self.state = "HALF_OPEN"        # time to test recovery
            else:
                raise CircuitOpenError()        # fail fast, no downstream call

        try:
            result = fn(*args)                  # attempt the call
        except Exception:
            self._on_failure()
            raise
        self._on_success()
        return result

    def _on_success(self):
        self.failures = 0
        self.state = "CLOSED"                    # recovered (or normal)

    def _on_failure(self):
        self.failures += 1
        if self.failures >= self.fail_max:
            self.state = "OPEN"                  # trip the breaker
            self.opened_at = time.time()

Design Decisions

Decision	Consideration
Failure threshold	Too low → trips on transient blips; too high → slow to protect
Cooldown duration	Too short → hammers a recovering service; too long → slow recovery
What counts as failure	Errors and timeouts (a slow call is a failure — always set timeouts)
Rolling window	Count failures over a recent window, not all time
Fallback	What to return when open (cached data, default, error)

Composing with Retry and Timeout

The circuit breaker is one of a trio of resilience patterns that work together (see Retry Pattern):

Every call:  wrap in a TIMEOUT (a hang counts as a failure)
Transient failure:  RETRY with backoff (a couple of attempts)
Persistent failure: the CIRCUIT BREAKER trips → stop retrying, fail fast

Retry WITHOUT a breaker is dangerous — it piles load on a failing service.
The breaker stops the retries once failure is clearly persistent.

Per-Dependency Breakers

Use a separate breaker per downstream dependency — one failing service shouldn't trip the breaker for a healthy one. This is closely related to the Bulkhead pattern (isolating failures so they don't spread).

Seeing It in Action

Scenario: Protecting a service that calls a flaky third-party API.

# One breaker PER dependency, with a timeout and a graceful fallback.
geocode_breaker = CircuitBreaker(fail_max=5, reset_timeout=30)

def geocode(address):
    try:
        return geocode_breaker.call(
            lambda: geocode_api.lookup(address, timeout=1.0)  # timeout = failure
        )
    except CircuitOpenError:
        # Breaker OPEN — don't even attempt the call. Serve a fallback.
        return cached_geocode(address) or approximate_from_zip(address)
    except (Timeout, APIError):
        return cached_geocode(address) or approximate_from_zip(address)

# Behavior over an incident:
#  Normal:            calls flow; failures near zero; breaker CLOSED.
#  API degrades:      timeouts/errors accumulate → after 5 → breaker OPEN.
#                     For the next 30s, calls fail FAST (no 1s hangs) and
#                     serve cached/approximate results → the app stays
#                     responsive and the third-party API isn't hammered.
#  API recovers:      after 30s → HALF-OPEN → a trial call succeeds →
#                     breaker CLOSES → normal service resumes automatically.

Why implementing it as a state machine matters: the circuit breaker's power comes from encoding a small, disciplined state machine that changes the system's behavior based on the recent health of a dependency — closed and trusting when things are fine, open and fail-fast when they're clearly broken, and cautiously half-open to probe for recovery. This is the State pattern applied to resilience, and getting the transitions and tuning right is what separates a breaker that protects the system from one that either trips on every transient hiccup (too sensitive) or fails to protect it in time (too tolerant). The critical implementation details — counting timeouts as failures (a hung call is worse than a fast error), using a rolling window rather than all-time counts, per-dependency breakers so one bad service can't disable a healthy path, and pairing the breaker with timeouts and bounded retries — are what make it work in practice. And the fallback is where the breaker connects to graceful degradation: an open breaker shouldn't just error, it should serve the least-bad alternative (cached or approximate results). Composed correctly, the pattern turns a failing dependency from a cascading, thread-exhausting outage into a fast, contained, self-healing degradation.

Interview Questions

1. Q: Describe the circuit breaker's three states and transitions. Hint: Closed (calls flow normally, failures counted; trips to Open when failures reach the threshold), Open (calls rejected instantly/fail-fast for a cooldown period, then moves to Half-Open), Half-Open (a few trial calls test recovery; success → Closed, failure → back to Open). It's the State pattern applied to resilience, letting the system auto-recover without hammering a still-broken dependency.

2. Q: Why must a circuit breaker treat timeouts as failures? Hint: A slow/hung call is often worse than a fast error — it ties up threads and connections while waiting, which is exactly the resource exhaustion the breaker exists to prevent. So every call must be timeout-bounded, and a timeout must count toward the failure threshold. Otherwise a dependency that's slow (not erroring) would never trip the breaker and would still exhaust the caller's resources.

3. Q: How do you tune the failure threshold and cooldown, and what happens if they're wrong? Hint: Failure threshold too low → trips on transient blips (unnecessary degradation); too high → slow to protect, damage done. Cooldown too short → repeatedly hammers a still-recovering service; too long → unnecessarily slow recovery. Use a rolling window (recent failures, not all-time) and tune to the dependency's real behavior — balancing responsiveness to real failures against tolerance of transient noise.

4. Q: Why use a separate breaker per dependency? Hint: So a failure in one downstream service doesn't trip the breaker for a healthy one — otherwise an unrelated dependency's problems would block calls to services that are working fine. Per-dependency breakers isolate failures (closely related to the Bulkhead pattern), keeping healthy paths available while only the genuinely failing dependency is short-circuited.

5. Q: How does the circuit breaker compose with retry and timeout? Hint: Timeout bounds each call (a hang counts as failure). Retry (with backoff/jitter) handles transient blips. The circuit breaker stops the retries once failure is clearly persistent, failing fast to protect resources and the struggling dependency. Retry without a breaker is dangerous — it piles load on a failing service; the breaker is what caps that. When open, serve a graceful fallback rather than erroring.

References

• Martin Fowler: CircuitBreaker — the canonical pattern description
• Release It! by Michael Nygard — origin of the stability pattern
• Topic 06 — Circuit Breaker — the availability-focused view

Dive Deeper

• Resilience4j CircuitBreaker — a production implementation and its configuration
• Polly (.NET) resilience patterns — circuit breaker + retry + timeout composed
• AWS Builders' Library: timeouts, retries, backoff — the companion patterns