The Java Virtual MachineMike Brunt

What is the JVM? Main JVM Suppliers ColdFusion and the JVM Java J2EE – Java EE Servlet Containers Where everything sits-fits Tuning the JVM Metrics Logging Using utilities to aid in tuning the JVM References Summary

Sun Microsystems created Java as a high-level programming language which should run on as many operating systems and hardware variations as possible. The goal of “write once run anywhere” meant that a predictable encapsulated operating system, no matter what the underlying hardware and software, was required.

The Java Virtual Machine (JVM or JRE) is a multi-threaded environment abstracting the actual computer operating system from the application code.

These are the main suppliers of Java Virtual Machines, there are others.

Sun Hotspot JVM IBM JVM BEA JRockit MAC OSRuntime for OSX Oracle JVM aka JServer – OJVM Microsoft JVM

Since ColdFusion MX 6.0-6.1, CF has been a Java Application which needs what is known as a Servlet Container to run on. The most popular Servlet Container for ColdFusion is JRun.

In the Enterprise edition of ColdFusion, the actual Servlet Container to be used can be selected during the install. In non-Enterprise installs JRun is automatically used. The main point being that there is always a Servlet Container.

This is a list 0f Java Servlet Containers which are certified as fully J2EE – Java EE compliant by Sun Microsystems.

Geronimo from Apache JBoss from Red Hat JRun from Adobe Oracle OC4J Sun Java System Application Server (Sun One). WebLogic Server from BEA WebSphere Server from IBM

This is a simple diagram showing where the Operating System, JVM, Servlet Container and Java Application Server sit in relativity.

Windows Install Windows Install

Standard-Server Enterprise

This is a simple diagram showing where the Operating System, JVM, Servlet Container and Java Application Server sit in relativity.

Garbage Collectors Garbage Collectors (different arguments to jvm.config) -XX:+UseParallelGC Use parallel garbage collection, this is typically the default

collector for ColdFusion installations. This collector is also referred to as the throughput collector. It uses a parallel version of the young generation collector. The old (tenured) generation is still cleaned with the default collector. Note This option can not be used in conjunction with -XX:+UseConcMarkSweepGC.

-XX:+UseConcMarkSweepGC Use concurrent garbage collection. This collector is also referred to as the concurrent low pause collector. It collects garbage in the old (tenured) generation concurrently to executing the application. Note that this option can not be used in conjunction with -XX:+UseParallelGC. Instead you may combine it with -XX:+UseParNewGC

-XX:+UseParNewGC Instructs the VM to use a parallel collector for the young generation. This option can be used in conjunction with -XX:+UseConcMarkSweepGC.

* As a note I have found that using the ConcMarkSweepGC along with ParNewGC can cause longer Full GC’s, I tend to use ParNewGC on it’s own.

Full Garbage Collections – Minimize Sun JVM’s use a generational garbage collector with two

young generations and an old generation. New objects are always allocated in the young generation part of the heap. Every time the young generation is full a minor garbage collection takes place. Minor garbage collections don't take very long.

Objects which survived some minor collections, because they are still referenced from the application, are promoted to the older generation. Usually the older generation is full at some point too, and needs to be cleaned. This leads to a major garbage collection or Full Garbage (Full GC) collection that removes garbage from both the old and the younger generations. Full GC’s are “stop-the-world” events and need to occur only when absolutely necessary.

Full Garbage Collections – Minimize One of the first methods I always want to employ when

troubleshooting is to ascertain how many Full GC’s are taking place. To do so I add these arguments to the jvm.config file, which is located in one of these directories depending on the installation:

Enterprise – {drive-volume}\jrun4\bin Standard/Server - {drive-volume}\CFusionMX7\runtime\bin-XX:+PrintGCDetails -XX:+PrintGCTimeStamps

-XX:+PrintHeapAtGC -verbose:gc -Xloggc:yourname.log*

*This will send Garbage collection output to the {std}-out.log and will create a log of all GC activity with whatever name you gave yourname in either..

Enterprise – {drive-volume}\jrun4\bin Standard/Server - {drive-volume}\CFusionMX7\runtime\bin

Full Garbage Collections – Minimize Here is an example of the Verbose Garbage Collection log output…

par new generation total 97088K, used 0K [0x10010000, 0x171d0000, 0x171d0000)

eden space 77696K, 0% used [0x10010000, 0x10010000, 0x14bf0000)

from space 19392K, 0% used [0x14bf0000, 0x14bf0000, 0x15ee0000)

to space 19392K, 0% used [0x15ee0000, 0x15ee0000, 0x171d0000)

tenured generation total 932096K, used 11109K [0x171d0000, 0x50010000, 0x50010000)

the space 932096K, 1% used [0x171d0000, 0x17ca95b8, 0x17ca9600, 0x50010000)

compacting perm gen total 26624K, used 26606K [0x50010000, 0x51a10000, 0x58010000)

the space 26624K, 99% used [0x50010000, 0x51a0b8a0, 0x51a0ba00, 0x51a10000)

} , 0.1523967 secs]

3691.704: [Full GC {Heap before GC invocations=8:

Heap

This Full GC was almost certainly unnecessary because at the time it ran only 1109k out of 932096k of the tenured (old) generation had been used. I also note the timestamp (3691.704) of every Full GC in the log. If I see a regular pattern it almost always means something is calling a Full GC perhaps using system.gc()

Full Garbage Collections – Minimize – How?

One pattern I have seen over and over with ColdFusion is that Full GC’s are very often occurring at regular 60 second intervals. Although I have never been able to pin this down fully, I suspect it is due to a Java process called RMI(Remote Method Invocation). So then what do we do to control these unnecessary “stop-the-world” Full GC’s? There are two alternatives…

1/ Disable Full GC’s using the -XX:+DisableExplicitGC jvm.config argument. I am not totally happy doing this and in many cases it will not work

2/ Run Full GC’s at longer regular intervals. Applying these arguments instead of -XX:+DisableExplicitGC will cause Full GC’s every 10 minutes (600,000 ms).-Dsun.rmi.dgc.client.gcInterval=600000-Dsun.rmi.dgc.server.gcInterval=600000

I have found that enabling metrics looking is invaluable in troubleshooting ColdFusion and JRun application issues. Metrics logging will show the status of the CF-JRun instance at pre-selected time intervals in terms of number of threads in use and available memory. In order to enable metrics logging we need to edit the jrun.xml file, here…

Enterprise install {drive-volume}\JRun4\servers\{servername}\SERVER-INFServer-Standard install{drive-volume}\CFusionMX7\runtime\servers\{servername}\SERVER-INF

Setting detail is on the next slide

Here is how to enable metrics logging in the jrun.xml file Firstly uncomment the service class…

Then set “metricsEnabled” to true, we will leave other settings to default.

Lastly I like to spilt the logs out to make reading them easier by this change

Sun JVM Versions up to 1.5 (5.0) http://java.sun.com/j2se/codenames.html

GC Viewer Tool (Sun JVM up to 1.4x)http://www.javaperformancetuning.com/tools/gcviewer/index.shtml

I have found that the only way to really determine the correct garbage collector to use for your applications and what the correct arguments are to pass to the JVM, via the jvm.config file, to effectively manage garbage collections is to first determine how many Full GC’s are occurring and control that if there are too many. Then load-test the application whilst tuning the arguments passed to the JVM, via the jvm.config file.

Mike Bruntmbrunt@go2ria.net