Our Payara Engineers have been working very hard on lots of new features ready for our final 5.181 release! One of the key features we intend to deliver is compatibility with MicroProfile 1.2, which will include (among other things) a Fault Tolerance API.
What is MicroProfile?
The Eclipse MicroProfile project is a collaboration between application server vendors and the community. The initial aim was to create a project that would allow developers with skill in Java EE to use that knowledge in creating modern cloud-native or microservice-based applications.
Fault Tolerance in Payara Micro (and Payara Server)
Along with the benefits of modularising and distributing a system across multiple instances, there also come new challenges. In a monolith, the whole system is either up or down. With microservices, part of the system can experience any number of problems which can propagate through the rest of the system in very unpredictable and subtle ways.
To mitigate this, there are a number of design patterns which have emerged to make microservice architectures more fault tolerant. These patterns have all been incorporated into the MicroProfile Fault Tolerance specification and include:
- Circuit Breaker
Offers a way to fail fast by automatically failing execution to prevent the system overloading and thereby prevent indefinite wait or timeout from clients
Isolate failures in part of the system while the rest of the system can still function.
Provide an alternative solution for a failed execution
Define a criteria for when and how often to retry
Define a duration for a timeout, after which execution is abandoned
A Worked Example
To demonstrate some of the fault tolerance behaviour available in Payara Micro (and Payara Server), I have created a simple example which demonstrates Retry, Fallback and Timeout which can be found in the payara/payara-examples project on GitHub, in the microprofile/fault-tolerance directory.
There are 2 examples of fault tolerance. One which demonstrates the
@Retry annotation to re-attempt an execution which has failed, and another which demonstrates a
@Timeout with a
@Fallback method to handle a slow invocation.
Build the Example
mvn clean install
Run the Example
To run the Uber JAR, run
java -jar target/fault-tolerance-1.0-SNAPSHOT-microbundle.jar
To run the WAR file on a different Payara Micro instance, run
java -jar /path/to/payara-micro.jar -deploy target/fault-tolerance-1.0-SNAPSHOT.war
Watch the output of Payara Micro to see the URLs created, and visit the endpoints to trigger an example of fault tolerance. Watch the logs to see how Payara Micro behaves.
How it works
The example is a simple JAX-RS class with two
getAllEmployees. There is a list of 4 people which can be returned.
There are 2 methods to introduce simple problematic behaviour, one
isDown() which simply returns a boolean based on
isSlow() which will sleep for a second based on
@Retryannotation has been added to the
getEmployeeById, where the
isDown()check may cause a
There is a
retryCounter to show the total amount of times the method has been called, so we can see the retry in action. The
isDown() method should cause failures around 80% of the time but, with 4 retries, this probability is reduced. If the Retry eventually gets a success, then the result will be returned and the user will not see any problem. If the Retry gets a failure every time, then it will fail as normal and the user will see an ungraceful exception.
An example of the log output for one test is below. I invoked the method 3 times, and have indicated this among the log messages:
In this example, the first invocation worked. The second did not work, and the Retry began to work, as shown in the log message. The method is then tried 4 more times and, on the 3rd attempt (number 5 shown by the counter), the method returned successfully. Next, I tried the invocation again and found that the method did not succeed, even after 4 retries, so an exception was thrown.
Timeout and Fallback
@Fallback annotations have been added to
getAllEmployees(), and an extra method added called
getAllEmployeesFallback() to handle invocations when the
@Timeout is triggered.
In this example, the method will sporadically sleep for longer than the configured timeout. In this case, the
@Fallback will intercept the invocation and call
getAllEmployeesFallback() to give the user a different message.
There are minimal configuration options for Fault Tolerance through asadmin commands. The only configurable aspect relates to the
@Asynchronous annotation, which will ensure that the execution of the client request will be on a separate thread. There are two asadmin commands which allow you to get and set the ManagedExecutorService name or ManagedScheduledExecutorService name as follows:
Fault Tolerance is just one of several MicroProfile 1.2 specs that are already implemented, with more on the way.
Stay tuned for more updates on using MicroProfile in Payara Server/Micro!