Circuit Breaker & Bulkhead Patterns

Circuit breaker: three states

1. Closed (normal). Calls go through as usual. The breaker counts successes, failures and slow calls in a rolling window. A rolling window is the most recent set of calls, such as the last 100 calls or the last 10 seconds.

2. Open (tripped). When failures cross a threshold (a limit; for example, "more than 50% failed, with at least 20 calls in the window"), the breaker opens. Now every call fails immediately, without contacting the dependency. The app returns a fallback (a simpler backup answer) or a fast error.

• The dependency gets space to breathe and recover, because no pile of requests hits it.
• Your service does not waste threads on waiting for timeouts.

3. Half-open (testing). After a cool-down period (a wait time; for example, 30 seconds), the breaker lets a few test calls through.

• If they succeed, the breaker goes back to closed.
• If they fail, it goes back to open for another cool-down.

What counts as a failure?

• Errors: exceptions, 5xx responses, connection failures.
• Timeouts.
• Slow calls. Many libraries can trip on "more than X% of calls slower than 2 seconds", because slowness is often the first sign of trouble.
• Usually not client errors like 400 or 404. Those mean that the request was wrong. They do not mean that the dependency is unhealthy.

Fallbacks: what to do when the circuit is open

A circuit breaker is only as good as its fallback. A fallback is what you do instead when the real call is not possible:

Never fake success for critical operations. A fallback should degrade the experience in an honest way (give a simpler result), and not hide a real failure (post 44).

// Resilience4j-style example (Java)CircuitBreaker cb = CircuitBreaker.of("reviews", CircuitBreakerConfig.custom()    .failureRateThreshold(50)                          // open at 50% failures    .slowCallRateThreshold(50)    .slowCallDurationThreshold(Duration.ofMillis(800))    .minimumNumberOfCalls(20)    .waitDurationInOpenState(Duration.ofSeconds(30))    // cool-down    .permittedNumberOfCallsInHalfOpenState(5)    .build());Supplier<List<Review>> call = CircuitBreaker.decorateSupplier(cb, () -> reviewsClient.get(productId));List<Review> reviews = Try.ofSupplier(call).getOrElse(Collections.emptyList());   // fallback

Bulkheads: isolate resources

Without bulkheads, one shared pool serves every dependency. The pool is limited, so it can run out:

With bulkheads, each dependency has its own limit:

Reviews can only ever use its 30 slots. When they are full, extra review calls are rejected at once (and go to the fallback), and pricing and stock keep working.

Ways to build bulkheads:

• Separate thread pools or concurrency limits (semaphores) per dependency. A semaphore is a counter that allows only a fixed number of calls at the same time.
• Separate connection pools per database or downstream service (a downstream service is one that you call).
• Separate queues and workers per job type (like the priority queues in post 38).
• Separate instances or clusters (groups of servers) for critical and non-critical traffic. For example, a dedicated set of servers for checkout, kept apart from browsing traffic.
• Per-tenant limits (a tenant is one customer or one customer account), so one noisy customer cannot use all the capacity (like the cells and shuffle sharding in post 40).
• Kubernetes resource limits per container, so one runaway service cannot eat all the CPU and memory of a node (a node is one machine in the cluster).

How the patterns work together

• Timeouts limit each call.
• Retries handle short problems.
• Circuit breakers stop calls to a clearly broken dependency.
• Bulkheads stop any one dependency from exhausting shared resources.
• Fallbacks keep the user experience as good as possible.

Where to put them

• In application code, with libraries: Resilience4j (Java), Polly (.NET), opossum (Node.js), pybreaker (Python) and gobreaker (Go).
• In the network layer, with a proxy or a service mesh. A proxy is a server that passes traffic on. A service mesh is a set of proxies that manage traffic between your services. Envoy, Istio and Linkerd offer circuit breaking (limits on connections and requests) and outlier detection, which removes single unhealthy hosts from load balancing for a while. You need no code changes, but it knows less about your business fallbacks.

Many systems use both: the mesh for general protection, and the app for smart fallbacks.

Pitfalls

• Breakers that hide problems. An open breaker with a quiet fallback can hide an outage for hours. Send an alert when a breaker opens.
• Wrong scope. One breaker for a whole service can trip because of one bad host. A breaker for each host may miss a problem that affects the whole service. Choose with care (outlier detection often works per host).
• Thresholds too sensitive, so the breaker trips on normal small errors. Use a minimum number of calls and rolling windows.
• Thresholds too loose, so the breaker does not trip in time to help.
• No fallback, so an open breaker only returns errors faster. That is still better than hanging, but plan real fallbacks for non-critical features.
• Bulkheads with the wrong size. If they are too small, you reject requests during normal busy times. Work out the size with Little's Law (post 8): concurrency ≈ requests per second × latency. (Latency is how long one call takes. Concurrency is how many calls run at once.)