In the previous blog of this series, we learned different ways to troubleshoot Java EE application. This blog will continue to focus on different ways and techniques to catch potential issues in the early stages and how to find the root cause of application performance issues.
Monitoring is an essential function in production environments that helps to ensure optimal performance and service uptime. In production, Operations teams wants to gather information about container to proactively check CPU throughput, Thread counts, Memory and Disk IO usage and view that data in a realtime dashboard (e.g Prometheus). To get a complete visibility into the applications and different components, Operations teams also need information about application e.g number of requests served, request duration, or the number active requests, fault tolerance responses, and several other metrics to find any potential problems in the early stage. So If the monitoring system raises an alarm because some resource is almost depleted, action can be taken to mitigate that problem.
The goal of Microprofile Metrics is to provide the monitoring support for applications as well as the underlying runtime and the metrics data are made available at the HTTP endpoints ‘/metrics’ with JSON and Prometheus format.Using a web browser to visit http://localhost:8080/metrics will result in an output of Prometheus text format as shown below:
The Prometheus server (an open-source leading monitoring solution) can scrapes the metrics data at an interval from ‘/metrics’ endpoint and store them in a time-series database, so you may extract and analyze metrics information in the Prometheus UI. Because it's Prometheus compatible, it's also well placed to use in a Kubernetes environment.
The MicroProfile Metrics configuration page can be found under Configurations → <config-name> → MicroProfile → Metrics tab.
MicroProfile Metrics is also configurable through asadmin commands. The set-metrics-configuration asadmin command is used to configure MicroProfile Metrics.
Find out more information about set-metrics-configuration asadmin command here.
MicroProfile Metrics specification defines 3 types of scopes for metrics :
Base: JVM statistics specific metrics, that every compliant vendor has to support.
Vendor: optional vendor-specific metrics that are not portable between different implementations.
Application: deployed applications specific metrics.
MicroProfile Metrics provides a way to register application-specific metrics either using Java Annotation or using programmatic api :
- @Counted - simple incrementing and decrementing counter.
- @Timed - tracks the duration of an event.
- @Metered - measures the rate at which a set of events occur.
- @Gauge - instantaneous reading of a particular value.
In most cases, it is enough to just provide one of the annotation @Counted, @Gauge etc, on the method or field you want to expose and the implementation will do the rest for you.
For explicit registration of metrics, MetricRegistry api can be used which maintains a collection of metrics along with their metadata. The metric registry can be obtained via CDI injection.
The MetricRegistry class has methods that can retrieve or register metrics using programmatic api :
Using annotation or programmatic api is few lines of code that provide a huge amount of insight into an application running in production.
The Microprofile Opentracing specification defines behaviors that allows you to trace the flow of a request across service boundaries. This is essential in a microservices environment where a request typically flows through the multiple services. By default tracing is enabled for all JAX-RS resources. To enable/disable tracing explicitly, the JAX-RS resource or non-JAX-RS class/method can be annotated with @Traced annotation. The annotation provides just two options: whether the method should be traced (the default value option of the annotation), and what the created Span should be called (operationName):
Now if the “/hello” endpoint is called, the following similar JSON representation should be available in log (if service-log notifier enabled) :
Payara Platform is using the existing Request Tracing Service feature for MicroProfile OpenTracing implementation. The Request Tracing Service configuration page can be found under Configurations → <config-name> → Request Tracing. To enable Request Tracing Service select Enabled :
Aside from this configuration settings, you can also define which notifiers will be used to relay the request tracing events by moving them to the Selected Notifiers box.
Request Tracing Service is also configurable through asadmin commands. By default, Request Tracing Service is disabled. The set-requesttracing-configuration asadmin command is used to configure Request Tracing Service.
Find out more information about set-requesttracing-configuration asadmin command here.
REST Monitoring Service exposes JMX MBeans over standard HTTP connection with a REST API in a JSON format, to stop JMX monitoring requiring RMI. The REST monitoring is hosted on the admin-listener (port 4848).
The REST Monitoring configuration page can be found under Configurations → <config-name> → Monitoring → REST. By default, REST monitoring is disabled. To enable REST Monitoring select Enabled :
REST monitoring is also configurable through asadmin commands. By default, REST monitoring is disabled. The set-rest-monitoring-configuration asadmin command is used to configure REST monitoring.
Find out more information about set-rest-monitoring-configuration asadmin command here.
The REST API is hosted on the following URL: http://<HOSTNAME>:<ADMIN_PORT>/<CONTEXT_ROOT>/rest
Which by default is http://localhost:4848/rest-monitoring/rest.
To read the particular MBean "java.lang:type=Threading", make a GET request to
which will result in a JSON representation of MBean similar to the following output:
You can also get the individual attributes returned by a composite MBean. So for the output above you can get a breakdown of each attribute in the array "value". For example, going to
returns the following response :
REST Monitoring can be used to keep track of things like memory pool usage or thread count when looking for bottlenecks or other potential issues. This helps you to make better decisions about how to maintain & optimize server resources and performance-tune code.
Additional Rest Monitoring Resources:
I hope you have enjoyed this tutorial and that it’s given you a decent amount of confidence to troubleshoot your application in production!
Want more detailed in-production guides?
Visit our resources page.