Tuesday, August 12, 2014

HotSpot: Monitoring and Tuning Metaspace in JDK 8

If you upgrade from JDK 7 to JDK 8, there are some changes in JVM that you should pay attention to.  One of them is:
  • The removal of Permanent Generation (PermGen) space.[1]  
Similar to the Oracle JRockit and IBM JVM's, the JDK 8 HotSpot JVM is now using native memory for the representation of class metadata, which is called Metaspace.  This may have removed the old OOM Error.[2] But ...

As described in [3], proper monitoring and tuning of the Metaspace is still required in order to limit the frequency or delay of metaspace garbage collections.  In this article, we will show how to achieve that.

Tuning Metaspace


When space (either PermGen or Metaspace) is filled up, class need to be unloaded.  In the PermGen era, when PermGen is filled up, a Garbage Collection (or GC) would occur and unload classes.  Without PermGen, we still need to have some way to know when to do a GC to unload classes.  This is when
-XX:MetaspaceSize=<size>

comes into play.   When the amount of space used for classes reaching MetaspaceSize, a GC is done to see if there are classes to be unloaded.  If you know that you are going to have lots of classes loaded and use more than 20M (this is the default value for my platform) class data, increasing MetaspaceSize can delay the GC's that are done to check for class unloading.  During your upgrade from JDK 7 to 8, as a first estimate, you can use the old value of PermSize to be your new MetaspaceSize.  You can also read [4] for more details.

Since Metaspace uses native memory, class metadata allocation theoretically is limited by amount of available native memory (capacity will of course depend if you use a 32-bit JVM vs. 64-bit along with OS virtual memory availability).  However, if you want to limit the amount of native memory used for class metadata, you can set:
-XX:MaxMetaspaceSize=<size>
which can restrict the extension of metaspace to the maximum at runtime.

How Many Classes Are Loaded?


There are multiple way[6,7] to find out how many classes are loaded and how much space they would take.  Here we introduce one way of finding that information.

jps is a Java tool that  you can use to  list the instrumented HotSpot Java Virtual Machines (JVMs) on the target system.  For example, here are the list of JVMs that are running on our Linux.

$ jps -l

32520 weblogic.Server
8936 weblogic.Server
11598 weblogic.Server
32646 weblogic.Server
7591 weblogic.Server
27429 weblogic.Server

To gather class information, you can use jstat tool which displays performance statistics for an instrumented HotSpot Java virtual machine (JVM).  For example, jstat has attached to process 27429 and displayed the statistics on the behavior of the class loader as below:

$ jstat -class 27429 1000 6

Loaded  Bytes  Unloaded  Bytes     Time
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22


Class Loader Statistics
ColumnDescription
LoadedNumber of classes loaded.
BytesNumber of Kbytes loaded.
UnloadedNumber of classes unloaded.
BytesNumber of Kbytes unloaded.
TimeTime spent performing class load and unload operations.
  

From the above output, we know that 46167 classes were loaded and they have taken 96M.  At the time of snapshots, HotSpot has also unloaded 406 classes from which 1.46 M were freed.  Based on this information, you can then decide how to set your MetaspaceSize and MaxMetaspaceSize (not advised) as described above.  However, be warned that once you have set MaxMetaspaceSize, you may still run into OOM error if metaspace cannot be extended further.

Acknowledgement


Some writings here are based on the feedback from Jon Masamitsu. However, the author would assume the full responsibility for the content himself.

References

  1. Java 7 features - PermGen removal
  2. 64-bit java.lang.OutOfMemoryError: PermGen space
  3. Java 8: From PermGen to Metaspace
  4. HotSpot: Understanding Metaspace in JDK 8 
  5. Latest JDK 8 and JDK 7 available here
  6. JavaOne Wednesday: Permanent Generation Removal Overview  
  7. jstat Tool: New Metaspace Statistics from -gc Option 
  8. VM Class Loading
  9. Other JDK 8 articles on Xml and More
  10. Tuning that was great in old JRockit versions might not be so good anymore
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

No comments: