Making Use of Payara Server's Monitoring Service - Part 1: Setting up the Service

(note: there is an updated version of this blog post available here https://blog.payara.fish/making-use-of-payara-servers-jmx-monitoring-service-part-1-setting-up-the-service)

 With the release of version 4.1.1.163, Payara Server includes a JMX Monitoring Service (technical preview) which can be used to log information from MBeans to the server log. Using the Monitoring Service, you can monitor information about the JVM runtime such as heap memory usage and threading, as well as more detailed information about the running Payara Server instance. The information is logged as a series of key-value pairs prefixed with the string PAYARA-MONITORING:, making it easy to filter the output using tools such as Logstash or fluentd. 

In this blog series we're going to show you exactly how to use the new Payara Server Monitoring Service. First, we'll take a look at setting up the service - let's get started!

Configuring the Monitoring Service

The Monitoring Service is disabled by default, so it must be configured to get some metrics. A convenient way to configure the Monitoring Service is with the new asadmin command set-monitoring-configuration.

To view the current monitoring configuration there is the get-monitoring-configuration asadmin command. The service configuration is displayed first, followed by a table of MBean attributes that are being monitored.

Calling the get-monitoring-configuration command should show the following for the default configuration:

Configuration options

The Monitoring Service has four configuration options, each of which have their own default values:

Alongside these options, the configuration accepts properties which represent the MBean attributes to monitor. Each property can have name, value and description. The name is the MBean Attribute, the value is the MBean ObjectName and the description is some extra information that is printed as part of the get-monitoring-configuration asadmin command. Both the name and value fields are required, but description is completely optional. An example property might have the following:

name="HeapMemoryUsage" value="java.lang:type=Memory" description="JVM_Heap_Usage"

asadmin> set-monitoring-configuration --addproperty 'name=HeapMemoryUsage value=java.lang:type=Memory description=JVM_Heap_Usage' --enabled false

After running the command the get-monitoring-configuration command will show something like this:

Here two options have been used: 1. --addproperty This option adds the property with the name, value and description given to it. These should be passed with the option as shown in the aforementioned command. name and value must be provided here for a valid monitoring property. 2. --enabled This option is used to enable or disable the service and is a required option. false was given to it to retain the current value.

Note: The HeapMemoryUsage MBean attribute is composite, with multiple values stored within it. This means that by only providing name=HeapMemoryUsage the Monitoring Service will log all values stored in the attribute. As HeapMemoryUsage has values of committed, init, max and used these would all be included in the log message. If you wished to log just the value for used then you could provide name=HeapMemoryUsage.used and just the used value would be added.

In general, if an attribute is composite, providing name={$MBEANATTRIBUTE}.{$VALUE} will cause the Monitoring Service to attempt to extract the value when logging. In the previous example $MBEANATTRIBUTE would be HeapMemoryUsage and the $VALUE is used.

Deleting properties

If you no longer wish to monitor a property or gave an invalid one, there is a --delproperty option available. To remove a property from the Monitoring Service, the name attribute of the property to delete must be specified:

asadmin> set-monitoring-configuration --delproperty HeapMemoryUsage --enabled false

asadmin> set-monitoring-configuration --dynamic true --enabled true

If a non-empty monitoring configuration existed when Payara Server was started, the service should then start monitoring to the server.log file: