GlassFish to Payara Server Migration: Replacing the Monitoring Scripting Client

In the fourth part of our continuing series on alternatives for commercial Oracle GlassFish features we are looking at the JMX Monitoring Service & the Payara HealthCheck Service as possible replacements for Oracle's Monitoring Scripting Client.

Monitoring Scripting Client - what is it?

With the Monitoring Scripting Client, administrators and other operations staff can write JavaScript templates that enable monitoring probes to execute the following tasks:

• Track application performance characteristics
• Troubleshoot functional issues
• Follow the state of internal components and services
• Observe how enterprise application behave in a general manner.

These probes have minimal overhead and impact on the performance of a production environment when enabled. This feature can help identify performance bottlenecks, predict server failures and identify their root causes to avoid future issues. It adapts to specific diagnostic needs that are useful to have in production environments.

When these probes are enabled, clients can listen to all events that are provided for the standard GlassFish Server monitoring (JMX). The server also comes with a set of scripts samples that cover many use cases for monitoring common statistics.

The Monitoring Scripting Client comes packaged as a web application that uses Comet support to run the scripts written out by users.

How can it be replaced?

The purpose of the Monitoring Scripting Client is allowing users of the Oracle GlassFish Server to have more control over the monitoring statistics of the server and to write custom event listeners that react to specific conditions that trigger these events. Payara Server doesn't come with a bundled application that allows writing custom event listeners, however, there are some good alternatives that can help users improve their diagnostics in the same manner:

1. Payara Server comes with a JMX Monitoring Service that can be used to log monitoring statistics gathered from MBeans to the server's log output. These metrics are logged together in a single log message as a series of key-value pairs prefixed by the string PAYARA-MONITORING, making it easier to filter these statistics and report them in a simpler way.

This service must be enabled first using the asadmin set-monitoring-configuration subcommand first. You can use this command to enable the service and configure the interval frequency for the statistics to be logged:

asadmin set-monitoring-configuration --logfrequency 60 --enabled true

You can also configure the service to log specific attributes of the MBeans used to gather data, using the --addproperty option with the subcommand. For example, to only log the Heap usage for the JVM:

asadmin set-monitoring-configuration --addproperty 'name=HeapMemoryUsage value=java.lang:type=Memory' --enabled false

For more details, see our official guides for configuring the JMX Monitoring Service. We also recommend the following list of articles on using the ELK (ElasticSearch, LogStash and Kibana) stack to take advantage of the Monitoring Service:

• Setting up the Monitoring Service with Payara Server
• Integrating with Logstash and ElasticSearch
• Using Kibana to visualize the data

2. Payara Server comes with a HealthCheck Service that can be used to automatically and periodically inspect low level metrics and alert users when these metrics go beyond a certain threshold. These alerts are logged to the server's log output:

Info:   CPU:Health Check Result:[[status=GOOD, message='CPU%: 23.28, Time CPU used: 1 seconds 163 milliseconds'']']
Info:   GC:Health Check Result:[[status=GOOD, message='1 times Young GC (PS Scavenge) after 72 milliseconds'']']
Severe: MMEM:Health Check Result:[[status=CRITICAL, message='Physical Memory Used: 7 Gb - Total Physical Memory: 7 Gb - Memory Used%: 96.74%'']']
Info:   HEAP:Health Check Result:[[status=GOOD, message='heap: init: 124 Mb, used: 203 Mb, committed: 460 Mb, max.: 910 Mbheap%: 22.0%'']']

The HealthCheck Service allows you to monitor the following statistics:

• Machine's CPU Usage
• Machine's Physical RAM Usage
• JVM's Heap Memory Usage
• JVM's Garbage Collection Pauses
• Threads Hogging the CPU
• JDBC Connection Pools Free Ratio

Each of these metrics can be configured separately using the appropriate asadmin commands:

asadmin healthcheck-configure-service-threshold --serviceName=healthcheck-cpu --dynamic=true --thresholdCritical=95 --thresholdWarning=75 --thresholdGood=60

asadmin healthcheck-configure-service --serviceName=healthcheck-gc --enabled=true --dynamic=true --time=5 --unit=MINUTES

For more details, see our official guides on configuring the HealthCheck Service.

Developers at the Payara Team are working on enhancing the way Payara Server provides monitoring statistics to its users, so keep an eye out for more helpful monitoring features coming in the future!