重试是为了应付偶尔抖动的情况，以求更多地挽回损失。可是如果provider持续的响应时间超长呢?如果provider是核心路径的服务，down掉基本就没法提供服务了，那我们也没话说。 如果是一个不那么重要的服务，却因为这个服务一直响应时间长导致consumer里面的核心服务也拖慢，那么就得不偿失了。单纯超时也解决不了这种情况了，因为一般超时时间，都比平均响应时间长一些，现在所有的打到provider的请求都超时了，那么consumer请求provider的平均响应时间就等于超时时间了，负载也被拖下来了。而重试则会加重这种问题，使consumer的可用性变得更差。因此就出现了熔断的逻辑，也就是，如果检查出来频繁超时，就把consumer调用provider的请求，直接短路掉，不实际调用，而是直接返回一个mock的值。等provider服务恢复稳定之后，重新调用。

断路器（Circuit Breaker）模式就是为了防止在分布式系统中出现这种瀑布似的连锁反应导致的灾难。

Goland circuitbreaker包

Circuitbreaker provides an easy way to use the Circuit Breaker pattern in a Go program.

Circuit breakers are typically used when your program makes remote calls. Remote calls can often hang for a while before they time out. If your application makes a lot of these requests, many resources can be tied up waiting for these time outs to occur. A circuit breaker wraps these remote calls and will trip after a defined amount of failures or time outs occur. When a circuit breaker is tripped any future calls will avoid making the remote call and return an error to the caller. In the meantime, the circuit breaker will periodically allow some calls to be tried again and will close the circuit if those are successful.

Installation

go get github.com/rubyist/circuitbreaker

Examples

Here is a quick example of what circuitbreaker provides

错误次数超过阈值就实现断路，如下面就是错误超过10次就断路

// Creates a circuit breaker that will trip if the function fails 10 times
cb := circuit.NewThresholdBreaker(10)

events := cb.Subscribe()
go func() {
  for {
    e := <-events
    // Monitor breaker events like BreakerTripped, BreakerReset, BreakerFail, BreakerReady
  }
}()

cb.Call(func() error {
	// This is where you'll do some remote call
	// If it fails, return an error
}, 0)

Circuitbreaker can also wrap a time out around the remote call.

// Creates a circuit breaker that will trip after 10 failures
// using a time out of 5 seconds
cb := circuit.NewThresholdBreaker(10)

cb.Call(func() error {
  // This is where you'll do some remote call
  // If it fails, return an error
}, time.Second * 5) // This will time out after 5 seconds, which counts as a failure

// Proceed as above

Circuitbreaker can also trip based on the number of consecutive failures.

连续错误多少次就实行断路操作，如下就是连续错误10次后就断路操作

// Creates a circuit breaker that will trip if 10 consecutive failures occur
cb := circuit.NewConsecutiveBreaker(10)

// Proceed as above

Circuitbreaker can trip based on the error rate.

错误率达到多少时就实行断路操作，如下就是采样100次，错误率达到0.95时就实行断路

// Creates a circuit breaker based on the error rate
cb := circuit.NewRateBreaker(0.95, 100) // trip when error rate hits 95%, with at least 100 samples

// Proceed as above

If it doesn't make sense to wrap logic in Call(), breakers can be handled manually.

cb := circuit.NewThresholdBreaker(10)

for {
  if cb.Ready() {
    // Breaker is not tripped, proceed
    err := doSomething()
    if err != nil {
      cb.Fail() // This will trip the breaker once it's failed 10 times
      continue
    }
    cb.Success()
  } else {
    // Breaker is in a tripped state.
  }
}