Circuit breaker

The idea

A circuit breaker wraps the calls one service makes to another. When those calls keep failing or hanging, it stops making them for a while and fails at once instead, so the caller’s threads and connections stay free for everything else.

Think of it like this Like the breaker in a fuse box: when one circuit faults, it trips and cuts that circuit so the rest of the house keeps its power. Once the fault is fixed, the circuit is switched back on.

How it works

1. Closed

   Calls go through as normal. The breaker keeps a window of recent calls and counts how many failed or took longer than its timeout.

2. Open

   When the failed share of the window crosses the threshold (say half of the last 50 calls), it trips. Every call now fails immediately, without waiting, until the cooldown ends.

3. Half-open

   After the cooldown it lets calls through again as a test. If they succeed, it closes; if they still fail, it opens for another cooldown.

It cycles: Closed → Open → Half-open → Closed.

Where you meet it

• Resilience4j (Java)
• Polly (.NET)
• opossum (Node.js)
• Netflix Hystrix (retired, in maintenance since 2018)
• Envoy and Istio: outlier detection ejects a failing host; their “circuit breaking” settings cap connections and pending requests

At a glance

Did you know

A breaker trips after too many failures and fails fast, giving the dependency room to recover.

Rule of thumb

Pair it with a fallback, such as cached data, so users still see something useful.

Watch out

Trips too early (flapping) or too late (cascade).

Its settings in ScaleMySystems

What you can change on it in the lab, and what each number does.

Concurrency (slots)

Calls it can work on at once (per instance when autoscaling). Needed slots ≈ calls per second × seconds each takes. Extra calls wait in the queue, then are rejected.

Queue capacity

Calls allowed to wait for a free slot. A bigger queue absorbs bursts but adds waiting time; 0 rejects straight away.

Latency (p50) (ms)

Typical time one call takes here, before any calls it makes itself. Slots are held for this long.

Latency (p99) (ms)

The slowest 1% of calls. Callers must set their timeout above this or they cut off healthy calls.

Timeout to dependencies (ms)

How long it waits for each call it makes before giving up. Set it above the dependency’s p99 but low enough that a dead dependency does not hold every slot.

Trip at error share

The breaker opens when this share of the last window of calls failed. Open, it fails fast instead of waiting.

Cooldown (ms)

How long the breaker stays open before letting calls through again.

Window (calls)

How many recent calls the error share is measured over. Small windows trip on a blip.