Diagnosing OutOfMemoryError or Memory Leaks in JRockit

When you run into OutOfMemoryError or other memory-leak issues, generating a heap histogram or heap dump can help you diagnose the memory-bloating issues.

In [1], it lists the following Java Heap related problems:

• Exceeding max heap 
• The heap is full and cannot fit a new object
• Large allocation
• The new object is too large for the contiguous free space
• Native exhaustion
• There is not enough native heap for the requested object^[10]
• GC Starvation
• The heap is almost full and causing frequent garbage collection^[9]
• Optimization
• Heap utilization is higher than expected for the current number of users

In this article, we will discuss the following topics:

1. Heap histogram vs. heap dump (see also [8])
2. How to generate heap histogram or heap dump in JRockit
3. JVM options that are useful for heap analysis

Heap Histogram vs. Heap Dump

A heap dump is a snapshot of all the objects in the Java Virtual Machine (JVM) heap at a certain point in time. The JVM software allocates memory for objects from the heap for all class instances and arrays. The garbage collector reclaims the heap memory when an object is no longer needed and there are no references to the object. By examining the heap you can locate where objects are created and find the references to those objects in the source.  However, dumping of Java heap is time-consuming and lengthy in size.

On the other hand, heap histogram gives a very good summary of heap objects used in the application without doing a full heap dump. It can help you quickly narrow down a memory leak. This information can be obtained in several way:

• Attach a running process using the command jrcmd.
• Generate from a core file or heap dump

Note that we refer to heap histogram, heap summary, or heap diagnostics interchangeably in this article.

Generating Heap Histogram

A heap histogram can be obtained from a running process using the command:

• jrcmd 20488 heap_diagnostics

--------- Detailed Heap Statistics: ---------
30.0% 65027k   672176     -1k [C
10.2% 22119k   943754     -2k java/lang/String
10.0% 21592k   183036     -3k [Ljava/lang/Object;
 4.7% 10185k   434587     +0k java/util/HashMap$Entry
 4.7% 10114k    27029  -1254k [B
 4.5% 9783k   111539     +0k [Ljava/util/HashMap$Entry;
 1.9% 4075k    34777     +0k java/lang/Class
 1.9% 4058k    86590     +0k java/util/HashMap
 1.6% 3448k   147156     +0k javax/management/ObjectName$Property
 1.2% 2593k    82994     +0k java/util/LinkedHashMap$Entry
 1.1% 2398k    76765     +0k java/util/concurrent/ConcurrentHashMap$Segment
 1.0% 2215k     9311     +0k [I
 0.9% 1975k    18469     +0k [J

In the output, there is a "Detailed Heap Statistics" section, which shows the total size and instance count for each class type in the heap:

• The first column corresponds to the Class object type contribution to the Java Heap footprint in %
• The second column correponds to the Class object type memory footprint in K
• The third column correponds to the # of Class instances of a particular type
• The fourth column correponds to the delta - / + memory footprint of a particular type

As you can see from the above snapshot, the biggest data type is [C (i.e., character array) and java.lang.String. In order to see which data types are leaking, you will probably need to generate several snapshots, which you might be able to observe a trend that can lead to further analysis.

Generating Heap Dump

Heap dump is a file containing all the memory contents of a Java application. It can be generated via 

• jrcmd 20488 hprofdump

• Wrote dump to /.../appmgr/APPTOP/instance/debug/jrockit_20488.hprof

Then you can use various tools to load that file and look at various things in the heap: how much each kind of object is using, what things are holding onto the most amount of memory, and so on.  The size of heap dump file is proportional to the size of Java Heap and can be large.

Three of the most common tools are:

• jhat
• This is the original heap analyzer tool, which reads the heap dump and runs a small HTTP server that lets you look at the dump through a series of web page links.
• VisualVM ^[3]
• MAT ^[4,5]

Heap-Related JVM Options

When your JVM runs into OutOfMemoryError, you can set:

• -XX:+HeapDumpOnOutOfMemoryError

to get a heap dump after the heap is big and bloated just before the JVM dies.  Also, you can provide the following flags:

• -XX:HeapDumpPath=<path to the destination>
• -XX:+ExitOnOutOfMemoryError

Similarly, you can get a heap histogram instead of a full heap dump using^[7]:

• -XX:+HeapDiagnosticsOnOutOfMemoryError 
• -XX:HeapDiagnosticsPath=<path to the destination>

References

1. Java™ Memory: Understanding and Analyzing the Java Heap
2. Getting histogram from .hprof heap dump file
3. VisualVM
4. Memory Analyzer (MAT)
5. Diagnosing Java.lang.OutOfMemoryError (XML and More)
6. Troubleshooting Memory Leaks
7. Default Values of JRockit's VM Options (XML and More)
8. Diagnosing Heap Stress in HotSpot (XML and More)
9. A Case Study of java.lang.OutOfMemoryError: GC overhead limit exceeded (XML and More)
10. Memory Mapped File and IO in Java
• Memory used to load Memory mapped file is outside of Java heap Space.  If OOM is caused by memory-mapped files, you may want to reduce your Java heap allocation (i.e., reducing -Xmx).