Xml and More: Performance Tuning with Hotspot VM Option: -XX:+TieredCompilation

Updated (09/26/2014):

In the latest JDK 8 release,^[20] TieredCompilation is ON by default. Depending on platforms, default ReservedCodeCacheSize is bigger too. For example, on Linux 64-bit platforms, its default size is 240MB.

Hotspot has two JITs named c1 (i.e., client JIT) and c2 (i.e., server JIT)^[1,5,6]. The client JIT starts fast but provides less optimizations. So, it is used for GUI application. The server JIT starts more slowly but provide very good optimizations. The idea of tiered compilation is to get the best of both compilers, first JITs the code with c1 and then if the code is really hot to recompile it with c2.

The tiered server runtime is enabled with the following Hotspot VM options:

-server -XX:+TieredCompilation

In this article, we will show you how to tune code cache size (i.e., -XX:ReservedCodeCacheSize^[4]) if tiered compilation is enabled. With the latest Hotspot builds,^[19] we did see some performance improvement if tiered compilation is enabled. However, we also noticed that if your code cache size is not big enough, it may negatively impact your application's performance (i.e., Response Time).

PrintCompilation

Before any performance tuning, you would like to learn how JIT compiler behaves first. You can track the behavior of the JIT compiler with:

-XX:+PrintCompilation ^[4]

With this VM option, JVM will print message when a method is compiled as follows:

 66    4     n 0     java.lang.System::arraycopy (0 bytes)   (static)
 66    5       3     java.lang.String::hashCode (67 bytes)
 67    1       3     java.lang.String::equals (88 bytes)
 68    2       3     java.lang.String::charAt (33 bytes)
504   69   !         java.util.jar.Attributes::read (410 bytes)
522   70             java.util.jar.Attributes$Name::isValid (45 bytes)
522   69   !         java.util.jar.Attributes::read (410 bytes)   made not entrant
864   69   !         java.util.jar.Attributes::read (410 bytes)   made zombie

Without much ado, we will refer you to read [5] for detailed explanation of each field in the output. For our purpose, we are interested in finding out:

How many methods have been compiled at the beginning of, say, 10 minutes?

You can find out that by counting the lines in the log file:

grep :: <logfile> | wc -l

since the compilation lines all have "::" in them. Based on this information, you can estimate what size of code cache size should be for better performance. You can also use this command line option along with instrumenting the benchmark to indicate when it has completed the warm-up period. For example, if you are observing -XX:+PrintCompilation output during the measurement interval, then benchmark has not reached JIT compiler steady state.

ReservedCodeCacheSize

To improve application's performance, you can set the maximum code cache size:

-XX:ReservedCodeCacheSize=256m

when tiered compilation is enabled for your JVM. ReservedCodeCacheSize (and InitialCodeCacheSize) is an option for the (just-in-time) compiler of the Java Hotspot VM. Basically it sets the maximum size for the compiler's code cache. In 1.5.0_06 and earlier, the default was 1024MB for Solaris 64-bit and amd64. Now the default is 48MB for Solaris 64-bit, amd64, and -server x86.

In our experiments, we have measured the performance of ATG CRM Demo application with different maximum code cache sizes. Here are our findings:

	-TC 64M	+TC 64M	+TC 128M	+TC 256M	+TC 512M	+TC 1024M
Response Time (seconds)	0.236	Failed	0.496	0.227	0.226	0.223

TC: Tiered Compilation

nnnM: Reserved Code Cache Size

Conclusion

As the benchmark results show, you can tune your application's performance by enabling tiered compilation with appropriate setting of code cache size. To estimate how much code cache to reserve, you can connect to the JVM using jconsole and use the memory tab to see how much code cache is filled. The performance improvement may vary based on your JVM versions and system capabilities. But, before you do any fine tuning, read [9] first.

Finally, be warned that options that are specified with -XX are not stable and are not recommended for casual use. These options are subject to change without notice^[4].