How To

 

This page is a step-by-step guide to help you set up your first Zapcat agent in your Java application. It will show you how to make a graph of the time your application spends running Hotspot compilation. I bet very few people have actually measured the time you lose to the Hotspot compiler.


This tutorial shows you the basics of adding items to your Zabbix manually. Of course, for most systems you can just use the Java and Hibernate templates and skip a lot of the boring work.


Before You Begin

This tutorial assumes that you have a working installation of Zabbix somewhere and that you have an account that has configuration privileges on that installation.




This page does not provide much in the way of background or documentation. You may want to read the architecture page first, if you have never used Zabbix before.


This guide only discusses installing a Zapcat agent in a regular Java application. For Java-based web servers such as Tomcat you can use the Tomcat plugin, and for Openfire XMPP server, you can use the Openfire plugin.


Below is the setup that we will use for this tutorial. I will use my own Zabbix server for all the examples on this web site. The sample programs all run on my development machine.


I’m sure you will quickly see where you should substitute my IP addresses and host names with yours.


The Zapcat Agent

Download Zapcat from its Source Forge page. Download the ZIP file with the compiled class files. Unzip the archive and place the Zapcat JAR file somewhere on your project’s class path.


To start an agent, use the code shown below. All we do is create a Zabbix agent, do some work and when we are done we stop the agent again. The agent starts as a daemon thread that starts listening to port 10052.

import org.kjkoster.zapcat.Agent;

import org.kjkoster.zapcat.zabbix.ZabbixAgent;


public class SampleAgent {

    public static void main(String[] args) throws Exception {

        Agent agent = null;

        try {

            agent = new ZabbixAgent();


            // simulate lots of important work being done...

            Thread.sleep(Long.MAX_VALUE);

        } catch (Exception e) {

            e.printStackTrace();

        } finally {

            agent.stop();

        }

    }

}


In this example, we are starting the Zapcat agent in the main() function of our application. Many Java applications are built as a plug-in, rather than a stand-alone application with its own main() function. You can start the Zabbix agent from any place you like, as long as you remember that the Zabbix cannot access the data in your application until you have started the agent.


Now compile the code and start the program with the agent. Best leave it running so that Zabbix can find it easily when you test your configuration.


The Zabbix Server

With the agent compiled into the code of your project, you can now focus on the Zabbix server configuration for this agent. We will use Zabbix to create a graph of the time that Hotspot spends compiling and optimising your code.


In Zabbix we will create a host that represents the virtual machine of our application and an item on that host to collect the compilation time.


Log into the Zabbix console with an administrator account. Then select “Configuration”, “Hosts” and click the “Create Host” button.


For the name of the host, pick the name of your application. Enter the DNS name and IP address of the machine that your application runs on. For the port number, enter 10052. Finally press “save” to add the host to the list.


The next step is to add an item to this host with the JMX query we want to do on the application. Let’s first determine the JMX query that we want the agent to execute for us and then create the Zabbix item to do the request.


From the Java API documentation of the compilation managed bean, we learn that we have to use the following object name:

java.lang:type=Compilation


On that same page, we see that the compilation bean has a method named “getTotalCompilationTime()”. This means that the bean has a property named “TotalCompilationTime” that we can query through JMX. Note that the first letter after “get” is not lower case, as you might expect from working with Java bean properties before.


The general syntax for JMX queries through the Zapcat agent is as follows:

jmx[<object name>][<property name>]


If we fill in the information that we learned from the API documentation, we get the following Zapcat query.

jmx[java.lang:type=Compilation][TotalCompilationTime]


The data type of the requested data is Java’s long integer, but Zabbix knows about measuring time and the graph will be much more readable if we make use of that. If we tell it that we are using seconds and convert the measured value to seconds, Zabbix will happily convert the measured value to minutes or even days where appropriate.


With this query, we may configure a Zabbix item on the host we have just created. In Zabbix, select “Configuration”, then “Items”. From the “Group” pick list, choose “all” and from the “Host” pick list, choose the host that you have just created. Then press “Create Item”


Give the item a descriptive name and enter the JMX query we have just constructed in the “Key” field. Choose the floating point data type. Enter “s” in the “Units” field, for seconds. Select a custom multiplier and give it the value of 0.001, to convert the sampled milliseconds into seconds. Finally press “save” to save the item.


Once you save the item, Zabbix will start querying the agent and gather statistics. Wait a minute or two for Zabbix to get a few data points. Perhaps this is a good time to refill your coffee mug.


Now press “Monitoring”, “Latest data” and pick your host from the selector group. Click on the “Graph” link to see the data plot.


If you leave your application to run for a while, Zabbix will build up a nice picture of the compilation time. Below is a picture of the sample application after I left it running for a few minutes. The restart of the application is clearly visible at 21:22h, as the compiler time is reset.


Here is the same picture, but for a Tomcat web server that runs a tiny site somewhere. As you can see, it spends a little more time compiling code than the sample application does, but not all that much.


Next time you have a discussion about how Java is slow because it spends all this time recompiling code runtime, you will have some hard figures to back your argument.


Configure Zapcat

Zapcat is practically configuration-free. There are only a few items that you can configure; the port it listens on and the bind address. These items are configured using the following system properties.

-Dorg.kjkoster.zapcat.zabbix.port=10049

-Dorg.kjkoster.zapcat.zabbix.address=127.0.0.1


In this example we use port number 10049 and we bind only to the loopback interface, effectively limiting access to our Zapcat agent from the local host only.


Both are optional. When omitted, Zapcat will use port 10052 and bind to all available interfaces by default.


If you are using a 1.1.x version of Zabbix, you need to tell Zapcat to use the older version of the agent protocol. Add the system property show below to the command line of you Java application.

-Dorg.kjkoster.zapcat.zabbix.protocol=1.1


When setting the protocol, you may choose either the value “1.1” or the value “1.4”.


Lock Down Access

The Zapcat agent accepts connections from any host or IP address. To regulate what hosts can and cannot connect to your application Zapcat offers a white list feature.


The white list is enabled by the “org.kjkoster.zapcat.whitelist” property. In the example below, we configure Zapcat to reject all connections, except from the IP address 192.168.10.12. Note that we can have both IP addresses and host names in the white list.

-Dorg.kjkoster.zapcat.whitelist=192.168.10.12


In case you have more than one machine that need to connect to your application’s agent, simply enter them as a comma separated list, as shown below.

-Dorg.kjkoster.zapcat.whitelist=192.168.10.12,devbox.domain.com


The host names are resolved to IP addresses whenever a connection is made. However, the default behaviour of Java is to cache DNS look-ups in the JVM as long at it runs. This is documented in Sun’s discussion of network related properties.


This behaviour is fine for the general case, where IP addresses are not likely to change. However, in some cases hosts may move to different IP addresses, for example because it gets its IP address from a DHCP server. In that case, set the property “networkaddress.cache.ttl” to a suitable value.


Configure Log4J

You probably noticed that Zapcat logs all traffic to a log4j logger. Zapcat is very chatty and unless you want to debug Zapcat itself, you probably don’t want to read most of what it’s doing. To silence Zapcat, add the following logger settings to your log4j.xml.

<logger name="org.kjkoster.zapcat">

<level value="WARN" />

</logger>

 
photo: Mark Morcomhttp://www.sxc.hu/profile/morcomm
if you find this interesting, terrible or just would like to know more, e-mail memailto:kjkoster@kjkoster.org?subject=
web statistics
download zapcathttp://sourceforge.net/project/showfiles.php?group_id=209024
http://java-monitor.com/