Skip to content

Instrument a circuit breaker

Create a CircuitBreaker and call it from your breaker's own hooks. It records what the breaker did, and does not make any decision itself:

breaker, err := instrument.NewCircuitBreaker(mp.Meter("checkout"), "payments")
if err != nil {
    return err
}
defer breaker.Close()

breaker.Success(ctx)      // a request succeeded
breaker.Failure(ctx)      // a request failed
breaker.Trip(ctx)         // the breaker opened
breaker.ShortCircuit(ctx) // a request was rejected while open

Trip both counts the transition and sets the state to open.

Set the other states

Call SetState on the other transitions, or nothing will take the metric back out of open:

breaker.SetState(instrument.BreakerHalfOpen)
breaker.SetState(instrument.BreakerClosed)

SetState stores the value for the next collect, so it takes no context. If you use a go/transit resilience breaker, FromResilienceState(int(state)) converts its state. Unknown values read as closed.

Close it when the breaker goes away

Call Close when the breaker goes away. It unregisters the callback that reports the state. A process that creates breakers as it runs and never closes them leaks one callback each, for the life of the provider.

Use one name per breaker

The name becomes the value of circuit_breaker.name, and every distinct value is a new series, so keep names to a small fixed set, never a per-call value. RED, USE and cardinality says what happens otherwise.

Give each breaker its own name per meter, because every metric carries that name as circuit_breaker.name. Two breakers sharing a name overwrite each other's state.

Read the signals

Signal Query
Is it open? circuit_breaker.state, where 0 is closed, 1 is open and 2 is half-open
How often does it trip? the rate of circuit_breaker.trips
What is it costing? the rate of circuit_breaker.short_circuits

Metric names lists every metric and its Prometheus counterpart.