Introduction
Circuit Breaker is an important feature of micro-service systems. The whole system could be highly affected if one or more services are unavailable. Circuit breaking alleviates these issues by avoiding calling services while they are down. In this article, you will learn how to integrate this feature into a Spring Boot project using the Resilience4j package available in Spring Cloud.
Circuit Breaker Pattern
The circuit breaker pattern is based on three states: closed, open and half-open. We will briefly describe them.
Closed State
If we consider two interacting services, one calling the other, this is the state when both are up and functioning correctly. Circuit breaker implementation will keep checking the health of the called service, counting the number of failed calls, and eventually triggering a state change based on a configured threshold.
Open State
If the threshold is reached, the state changes to open. The call will then be bypassed by returning an exception (or another convenient result, if configured).
Half-open State
When a predefined time range is passed from the last open state, the state changes to half open. In this state, a limited number of calls is allowed: if the failing rate is still above the threshold it returns to open, otherwise it changes to closed.
Spring Cloud Circuit Breaker Abstraction
Spring Cloud abstracts the way to approach circuit breaking using a pluggable architecture. The developer is free to choose which implementation to use. The following are supported:
- Resilience4J
- Spring Retry
Using one or the other in a Spring Boot project is just a matter of setting a specific dependency. If we want to use Resilience4J in our project we need to set the spring-cloud-starter-circuitbreaker-resilience4j starter, after having defined the Spring Cloud release train in the dependency management section:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>2023.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>
When Spring Boot detects the presence of the Resilience4j package in the classpath, it activates its autoconfiguration features, with a set of defaults.
Resilience4j Features
Resilience4j is a library aimed at improving the fault tolerance of micro-service systems. Besides the circuit breaker functionality, It offers several other features:
- Circuit breaker: deals with remote not responding services by-passing the faulting calls
- Retry: implements a retry mechanism for the failing requests
- Time limiter: set a time limit for remote calls
- Rate limiter: set a limit to the number of requests in a given time
- Bulkhead: set a limit to the number of concurrent requests
Resilience4j Circuit Breaker Configuration
We have to edit the application.yaml file to configure the circuit breaker functionality in a Spring Boot application. This is an example:
resilience4j:
circuitbreaker:
instances:
CircuitBreakerApi:
sliding-window-size: '10'
sliding-window-type: count_based
failure-rate-threshold: '50'
minimum-number-of-calls: '5'
permitted-number-of-calls-in-half-open-state: '3'
wait-duration-in-open-state: 5s
automatic-transition-from-open-to-half-open-enabled: 'true'
We can define one or more instances with a given name (‘CircuitBreakerApi’ in the above example). That name can then be used to apply the feature to endpoints by a specific annotation.
sliding-window-size
It defines the size of a sliding window that constitutes a sample of the call outcomes. The circuit breaker uses this sample as a base to calculate the failure rate of an endpoint. There are two types of sliding windows, the count-based type and the time-based type. In the count-based type, the sliding window is implemented as a circular array with a size equal to sliding-window-size: each time a new call outcome is stored in the array, the oldest one is evicted.
In the time-based approach, the sliding window size is defined in seconds. In this case, the circular array represents a set of buckets. If the size is 10 seconds, we thus have 10 buckets, one bucket for each second in a consecutive interval of time in which the measurements are made. Each bucket stores the outcomes of all the calls made in a particular second in the time interval.
failure-rate-threshold
It defines the percentage of failed calls in a sliding window that makes the circuit breaker stop further requests.
minimum-number-of-calls
It represents the minimum number of failed calls that can be used in a sliding window to calculate the failure percentage.
permitted-number-of-calls-in-half-open-state
It’s the maximum number of calls in a half-open state. After that number, the state will pass to open it if those calls fail, otherwise it will pass to closed.
wait-duration-in-open-state
Time interval before going to half-open from the open state.
automatic-transition-from-open-to-half-open-enabled
If set to true, it automatically triggers the transition to the half-open state, where the wait-duration-in-open-state time interval is passed. If set to false, it triggers the transition only if the time interval is passed and a call is made (this avoids the need for a monitoring thread).
Resilience4j Other Features Configuration
In this article, we will cover only the circuit breaker, but just to complete the picture, we show here an example of the configuration of the other features:
resilience4j:
retry:
instances:
RetryApi:
max-attempts: '3'
wait-duration: 1s
timelimiter:
instances:
TimeLimiterApi:
timeout-duration: 2s
cancel-running-future: 'true'
bulkhead:
instances:
BulkheadApi:
max-wait-duration: '1'
max-concurrent-calls: '3'
ratelimiter:
instances:
RateLimiterApi:
timeout-duration: 0s
event-consumer-buffer-size: '50'
register-health-indicator: 'true'
limit-for-period: '5'
subscribe-for-events: 'true'
limit-refresh-period: 60s
allow-health-indicator-to-fail: 'true'
Testing Circuit Breaker
We will demonstrate how a Resilience4j circuit breaker works by using a Junit Test and two simple Spring Boot services. You can find the example in the link at the bottom of this article. The client component is meant to represent a book service of a library software that calls an author service to retrieve information about specific authors. For this test, the client service will expose a /getAuthorServiceInstance endpoint that, in turn, will remotely call a /getInstance endpoint that serves the sole purpose of pinging the server, returning a string with the service server name and port.
The example uses the following versions of Java, Spring Boot, and Spring Cloud:
- Spring Boot: 3.2.1
- Spring Cloud: 2023.0.0
- Java 17
The configuration application.xml file for the test environment contains the following settings:
resilience4j:
circuitbreaker:
instances:
CircuitBreakerApi:
sliding-window-size: '10'
minimum-number-of-calls: '5'
permitted-number-of-calls-in-half-open-state: '3'
failure-rate-threshold: '50'
wait-duration-in-open-state: 5s
sliding-window-type: count_based
automatic-transition-from-open-to-half-open-enabled: 'true'
The controller method of the client endpoint is annotated with the @CircuitBreaker annotation, using the instance name “CircuitBreakerApi”:
@GetMapping(value = "/getAuthorServiceInstance")
@CircuitBreaker(name = "CircuitBreakerApi", fallbackMethod = "getAuthorServiceInstanceFallback")
public String getAuthorServiceInstance() {
return bookService.getAuthorServiceInstance();
}
The annotation above defines a fallback method returning a simple string, which will be executed when the state is open, in place of the original call:
public String getAuthorServiceInstanceFallback(Exception ex) {
return "Fallback content";
}
We mock the /authors/getInstance service by using the WireMockExtension class and make it return a server error. We then execute a for loop with five calls to the /getAuthorServiceInstance client endpoint using a TesRestTemplate class instance. At each step, we will check the circuit breaker state using the instance obtained from the circuit breaker registry.
Since we have set the minimum-number-of-calls parameter to 5, we expect it will change the status from closed to open only at the fifth call. Then, if we continue to call the service, we expect the status to change from open to half-open and vice versa since the service is always down. You can see the test implementation in the following code snippet:
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
class CircuitBreakerTest {
@RegisterExtension
static WireMockExtension mockService = WireMockExtension.newInstance()
.options(WireMockConfiguration.wireMockConfig()
.port(8091))
.build();
@Autowired
private TestRestTemplate restTemplate;
@Autowired
private CircuitBreakerRegistry circuitBreakerRegistry;
@Test
public void testCircuitBreaker() {
mockService.stubFor(WireMock.get("/authors/getInstance")
.willReturn(serverError()));
Optional<CircuitBreaker> circuitBreakerOptional = circuitBreakerRegistry.getAllCircuitBreakers()
.stream()
.findFirst();
CircuitBreaker circuitBreaker = circuitBreakerOptional.get();
for (int i = 0; i < 5; i++) {
restTemplate.getForEntity("http://localhost:8080/library/getAuthorServiceInstance", String.class);
if (i < 4) {
assertTrue(circuitBreaker.getState()
.equals(CircuitBreaker.State.CLOSED));
} else {
assertTrue(circuitBreaker.getState()
.equals(CircuitBreaker.State.OPEN));
}
}
for (int i = 0; i < 5; i++) {
restTemplate.getForEntity("http://localhost:8080/library/getAuthorServiceInstance", String.class);
assertTrue(circuitBreaker.getState()
.equals(CircuitBreaker.State.OPEN) || circuitBreaker.getState()
.equals(CircuitBreaker.State.HALF_OPEN));
}
}
}
Conclusion
The circuit breaker pattern is essential in microservice systems. It allows us to minimize the impact of specific node failures on the whole system. Resilience4j, among other useful features, provides an implementation for this pattern that can be easily plugged into Spring Boot, using the Spring Cloud circuit breaker abstraction.
You can find the source code of the example in this article on GitHub.