JDK 8: When Softly Referenced Objects Are Flushed?

There are four different degrees of reference strength in Java language:[1]

1. Strong
• StringBuffer strongRef = new StringBuffer();
2. Soft
• SoftReference softRef = new SoftReference(obj);
3. Weak
• WeakReference weakRef = new WeakReference(widget);
4. Phantom
• PhantomReference phantomRef =new PhantomReference(bool,rq);

in order from strongest to weakest.

In this article, we will focus on softly referenced objects and when they will be flushed by Garbage Collector (GC) in HotSpot.

SoftReference vs WeakReference

A soft reference is exactly like a weak reference, except that it is less eager to throw away the object to which it refers. An object which is only weakly reachable (the strongest references to it are WeakReferences) will be discarded at the next garbage collection cycle, but an object which is softly reachable will generally stick around for a while.

SoftReferences aren't required to behave any differently than WeakReferences, but in practice softly reachable objects are generally retained as long as memory is in plentiful supply. This makes them an excellent foundation for a cache since you can let the garbage collector worry about both how reachable the objects are and how badly it needs the memory they are consuming.

When Softly Referenced Objects Are Flushed?

Prior to version 1.3.1, the Java HotSpot VMs cleared soft references whenever it found them. Starting with 1.3.1, softly reachable objects will remain alive for some amount of time after the last time they were referenced. The default value is one second of lifetime per free megabyte in the heap. This value can be adjusted using the -XX:SoftRefLRUPolicyMSPerMB flag, which accepts integer values representing milliseconds. For example, to change the value from one second to 2.5 seconds, use this flag:

-XX:SoftRefLRUPolicyMSPerMB=2500

When, exactly, is a soft reference freed?  Here are the factors to be considered:

• First the referent must not be strongly referenced elsewhere
• If the soft reference is the only remaining reference to its referent, the referent is freed during the next GC cycle only if the time it was last accessed is greater than (AmountOfFreeMemoryInMB * SoftRefLRUPolicyMSPerMB).
• AmountOfFreeMemoryInMB is calculated differently for server and client VMs:
• • • Server VM
    • Uses the maximum possible heap size (as set with the -Xmx option) to calculate free space remaining
    • -Xmx therefore has a significant effect on when soft references are garbage collected.
    • The general tendency is for the Server VM to grow the heap rather than flush soft references
    • Client VM
    • Uses the current heap size to calculate the free space
    • The Client VM will have a greater tendency to flush soft references rather than grow the heap

Conclusion

• When to use SoftReference?
• Soft references are used when the object in question has a good chance of being reused in the future,
• For example, as a cache implementation
• Soft references are essentially one large, least-recently-used cache
• Performance consideration
• The key to getting good performance from that cache is to make sure that it is cleared on a timely basis.
• Eagerly vs. lazily reclaimation of softly referenced objects
• You can tune how eagerly they are reclaimed by setting:
• -XX:SoftRefLRUPolicyMSPerMB flag
• Default is 1000 ms. Optimal setting depends on the applications and which GC is used.
• This behavior is not part of the VM specification, however, and is subject to change in future releases. Likewise the -XX:SoftRefLRUPolicyMSPerMB flag is not guaranteed to be present in any given release.

References

1. Understanding Weak References Blog (good)
2. Frequently Asked Questions About the Java HotSpot VM
3. HotSpot Virtual Machine Garbage Collection Tuning Guide
4. Garbage-First Garbage Collector Tuning
5. Other JDK 8 articles on Xml and More
6. Tuning that was great in old JRockit versions might not be so good anymore
   
   • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
   • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

G1 GC: What Is "to-space exhausted" in GC Log?

As shown in [1], you can enable basic garbage collection (GC) event printing using two options:

• -XX:+PrintGCTimeStamps (or -XX:+PrintGCDateStamps) 
• -XX:+PrintGCDetails

These additional printing incurs minimal overhead while provide useful information for understanding the healthiness of your running garbage collector (e.g., Generation First GC).

In this article, we will discuss the significance of to-space exhausted events in your GC logs.

What Is "to-space exhausted"?

When you see to-space exhausted messages in GC logs, it means that G1 does not have enough memory for either survived or tenured objects, or for both, and the Java heap cannot be further expanded since it is already at its max.^[5]  This event only happens in mixed GC 's when G1 tries to copy live objects from the source space to the destination space.  Note that, in G1, it avoids heap fragmentation by compaction (i.e., done when objects are copied from source region to destination region).

Example message:

5843.494: [GC pause (G1 Evacuation Pause) (mixed)... (to-space exhausted), 0.1145790 secs]

When that happens, GC pause time will be longer (i.e., since both RSet and CSet^[5] need to be re-generated) and sometimes it may require a Full GC to resolve the issue. 

A Case Study of "to-space exhausted" Events

Using a benchmark with large live data size,^[3] a "to-space exhausted" event can be triggered if there is not enough breathing room for shuffling live data objects in the Java heap.  For example, we have a test case which its live data set occupies 68% of the Java heap at run time.  In a 4-hour run, we have seen the following sequence of events:

5843.494: [GC pause (G1 Evacuation Pause) (mixed)... (to-space exhausted), 0.1145790 secs]
5848.831: [GC pause (G1 Evacuation Pause) (mixed)... (to-space exhausted), 0.0847160 secs]
5856.640: [GC pause (G1 Evacuation Pause) (mixed)... 0.0864060 secs]
5866.182: [GC pause (G1 Evacuation Pause) (mixed)... 0.0821220 secs]
5875.550: [GC pause (G1 Evacuation Pause) (mixed)... 0.0549090 secs]
5882.353: [GC pause (G1 Evacuation Pause) (mixed)... 0.0827630 secs]
5896.331: [GC pause (G1 Evacuation Pause) (mixed)... 0.0632660 secs]
5901.966: [GC pause (G1 Evacuation Pause) (mixed)... 0.5356580 secs]
5902.813: [GC pause (G1 Evacuation Pause) (mixed)... (to-space exhausted), 1.3480320 secs]
5904.162: [Full GC (Allocation Failure) ... 5.0413590 secs]

A series of "to-space exhausted" events eventually led to a Full GC.  As discussed in [4], adding -Xmn400m on the command line to the same test case further add insult to injury—instead of seeing 3 "to-space exhausted" events, now we saw 20 of them.  This is due to we have further cut down the breathing room for G1.

How to Deal with "to-space exhausted" Events?

"to-space exhausted" events happen when G1 runs out of space (either survivor space or old space; see diagram above) to copy live objects to.  When it happens, sometimes G1's ergonomics can get by the issue by dynamically re-sizing heap regions.  If not, it eventually leads to a Full GC event.

When "to-space exhausted" events happen, it could mean you have a very tight heap allocated for your application.  The easiest approach to resolve this is increasing your heap size if it's possible. Alternatively, sometimes it's possible to tune G1's mixed GC to get by (see [4]). As shown in the above section, we have a high live data occupancy (i.e., 68%) in Java heap.  However, after tuning mixed GC, we can get by with a couple of Full GC's in a 4-hour run, which yields satisfactory result.

Note that mixed GC can clean up both young and old regions. If it does not take an old region, it would be a young(-only) GC. G1's mixed GC's are meant as a replacement for Full GC's. Instead of doing all the work at once, the idea is to achieve the same task in steps. Each small steps in mixed GC is smaller than a Full GC. Because G1 does not intend to reclaim all garbage as in a Full GC event, mixed GC uses less time.  However, Full GC is a compaction event which it can walk around "to-space exhausted" issue without needing extra space.

Acknowledgement

Some writings here are based on the feedback from Thomas Schatzl and Yu Zhang. However, the author would assume the full responsibility for the content himself.

References

1. g1gc logs - basic - how to print and how to understand
2. g1gc logs - Ergonomics -how to print and how to understand
3. JRockit: How to Estimate the Size of Live Data Set
4. JDK 8: Is Tuning MaxNewSize in G1 GC a Good Idea?
5. Garbage First Garbage Collector Tuning
• Note that "to-space overflow" message was removed and only "to-space exhausted" remains.
• RSet (Remembered Set)—Tracks object references into a given region. There is one RSet per region in the heap. The RSet enables the parallel and independent collection of a region. The overall footprint impact of RSets is less than 5%.
• CSet (Collection Set)—Are the set of regions that will be collected in a GC. All live data in a CSet is evacuated (copied/moved) during a GC. Sets of regions can be Eden, survivor, and/or old generation. CSets have a less than 1% impact on the size of the JVM. 
6. HotSpot Virtual Machine Garbage Collection Tuning Guide
7. Garbage-First Garbage Collector Tuning
8. Getting Started with the G1 Garbage Collector
9. Garbage-First Garbage Collector (JDK 8 HotSpot Virtual Machine Garbage Collection Tuning Guide)
10. Other JDK 8 articles on Xml and More
11. Concurrent Marking in G1

JDK 8: Is Tuning MaxNewSize in G1 GC a Good Idea?

For a throughput GC such as Parallel GC, it often helps performance by setting MaxNewSize (or -Xmn).  For G1 GC,^[1] is it the same? The short answer is yes/no and it depends (confession: I said "no" because I haven't done enough tests to exclude it).

In this article, we will demonstrate one case that setting MaxNewSize for G1 actually hurts the application's performance.  BTW, after benchmarking G1 in JDK 7, another author has claimed that:^[14]

• There is little to gain and much to lose from setting the new generation size explicitly.

Live Data Size

To tune the heap size of a Java Application, it's important to find out what its live data size is.  To learn how to estimate a Java application's live data size, read [2].  For the benchmark we are using, its live data size is around 1400 MB. With this benchmark, we are interested in knowing how G1 performs with MaxNewSize set to 400MB and Java heap set to 2GB.

Regularly we use the following settings for the G1 tuning:

• -Xms2g -Xmx2g -XX:+UseG1GC

plus some options for mixed GC (read [3]). So, for this experiment, we have added MaxNewSize setting like this:

• -Xms2g -Xmx2g -XX:+UseG1GC -Xmn400m

This has turned out hurting the performance a lot. Note that -Xmn400m is the optimal setting of our benchmark if Parallel GC is used.

Concurrent Cycle in G1 GC

G1 has four main operations:^[5]

• A young collection
• A background, concurrent cycle
• A mixed collection
• If necessary, a full GC

Without much ado, read [5-9] for the needed background of G1 GC.

G1 is a concurrent collector.  Its concurrent cycle has multiple phases—some of which stop all application threads (denoted by STW) and some of which do not:

• Initial-mark (STW)
• Denoted by: 
• [GC pause (G1 Evacuation Pause) (young) (initial-mark)
• [GC pause (Metadata GC Threshold) (young) (initial-mark)
• The G1 GC marks the roots during this phase. This phase is piggybacked on a normal (STW) young garbage collection.
• Root Region Scan
• Denoted by: 
• [GC concurrent-root-region-scan-start]
• [GC concurrent-root-region-scan-end
• The G1 GC scans survivor regions of the initial mark for references to the old generation and marks the referenced objects. 
• This phase runs concurrently with the application (not STW) and must complete before the next STW young garbage collection can start.
• Concurrent marking
• Denoted by:
• [GC concurrent-mark-start]
• [GC concurrent-mark-end
• The G1 GC finds reachable (live) objects across the entire heap. 
• This phase happens concurrently with the application, and can be interrupted by STW young garbage collections.
• Remark (STW)
• Denoted by:
• [GC remark
• This phase is STW collection and helps the completion of the marking cycle. 
• Finds objects that were missed by the concurrent mark phase due to updates by Java application threads to objects after the concurrent collector had finished tracing that object.
• G1 GC drains SATB buffers, traces unvisited live objects, and performs reference processing.
• Cleanup (STW)
• Denoted by:
• [GC cleanup 1936M->1931M(2048M)
• Prepare for next concurrent collection by clearing data structures.
• In this final phase, the G1 GC performs the STW operations of accounting and RSet scrubbing. 
• During accounting, the G1 GC identifies completely free regions and mixed garbage collection candidates. The cleanup phase is partly concurrent when it resets and returns the empty regions to the free list.
• Concurrent cleanup:
• Denoted by:
• [GC concurrent-cleanup-start]
• [GC concurrent-cleanup-end
• In this phase, G1 reclaims regions which were found empty during marking.
• Adds empty regions to the free list—i.e., thread local free lists are merged into global free list

G1 GC uses the Snapshot-At-The-Beginning (SATB) algorithm, which takes a snapshot of the set of live objects in the heap at the start of a marking cycle.  During the concurrent cycle, young garbage collections are allowed, which are triggered when eden fills up (note that initial-mark is implemented using a young collection cycle) .

The set of live objects after marking cycle is composed of the live objects in the snapshot, and the objects allocated since the start of the marking cycle. The G1 GC marking algorithm uses a pre-write barrier to record and mark objects that are part of the logical snapshot.

After the concurrent cycle, we expect to see:^[5]

• The eden regions before the marking cycle have been completely freed and new eden regions have been  allocated
• Old regions could be more occupied because the promotion of live objects from young regions
• Some old regions are identified to be mostly garbage and become candidates in later mixed or old collection cycles

Diagnosis

The reason of setting -Xmn400m has caused the regression of G1 is:

The option has forced G1 to use a young generation space of up to 400 MB and leaves G1 just 200 MB (i.e. 2048MB - 400MB - 1400MB) breathing room for shuffling live objects around.    

The small free space has negative impact on G1's concurrent cycles, which has caused most marking cycles not being completed in time.  

Before setting -Xmn400m on the command line, we have found only one instance of aborted marking cycle, which is denoted by:

• [GC concurrent-mark-abort]

After setting it, we have seen marking cycle was aborted 105 times out of 150 instances.  The lesson we have learnt here is that for a tight heap like our benchmark, it's not a good idea to set -Xmn400m for G1.  Without setting MaxNewSize, G1's ergonomics will dynamically adjust young generation space and it turns out that G1 can do a better job in this case.

Differences between Parallel GC and G1 GC

Note that -Xmn400m is the optimal setting for our benchmark if Parallel GC is used.  When we set MaxNewSize to be 400MB for G1, our benchmark regressed.  So, what's the difference between G1 GC and Parallel GC?

Initially, G1 GC was designed to replace CMS ^[10] for its relatively lower and more predictable pause times.  This is different from the design of Parallel GC which aims for higher throughput.  To help gain higher throughput, Parallel GC tries to:

• adjust young generation as large as possible and let it run into a full GC
• In our 4-hour experiments, we usually see around 50 full GC's in Parallel GC
• However, for G1's performance tuning, we want to avoid full GC's (see [11] for how)
• Which means more frequent full GC is expected
• Note that Parallel GC's Full GC pause time is shorter than G1's in the current JDK 8 releases

To conclude, if you upgrade JDK or switch garbage collector,^[7] always re-evaluate original optimal VM settings.

Acknowledgement

Some writings here are based on the feedback from Thomas Schatzl. However, the author would assume the full responsibility for the content himself.

References

1. Garbage First Garbage Collector Tuning
2. JRockit: How to Estimate the Size of Live Data Set
3. g1gc logs - basic - how to print and how to understand
4. g1gc logs - Ergonomics -how to print and how to understand
5. Java Performance: The Definitive Guide (Strongly recommended)
6. Garbage First Garbage Collector Tuning - Oracle
7. Our Collectors by Jon Masamitsu
8. Understanding Garbage Collection
9. HotSpot VM Performance Tuning Tips
10. Understanding CMS GC Logs
11. G1 GC: Tuning Mixed Garbage Collections in JDK 8
12. G1 GC Glossary of Terms
13. Learn More About Performance Improvements in JDK 8 
14. Benchmarking G1 and other Java 7 Garbage Collectors
15. HotSpot Virtual Machine Garbage Collection Tuning Guide
16. Getting Started with the G1 Garbage Collector
17. Garbage-First Garbage Collector (JDK 8 HotSpot Virtual Machine Garbage Collection Tuning Guide)
18. Other JDK 8 articles on Xml and More
19. Tuning that was great in old JRockit versions might not be so good anymore
    
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

New JDK 8 support in WebLogic Server 12.1.3.0

Oracle has just announced that Oracle WebLogic Server 12.1.3 has been certified on Java SE 8. It is supported on:

• Windows
• Linux
• Solaris 64-bit 

platforms with HotSpot JDK8 Update 20 or later.

If you want to learn more about what is supported and what are the limitations, check:

• What's New in Oracle WebLogic Server
• WebLogic Server 12.1.3 on JDK 8.
• FMW 12.1.3 Certification Matrix.
• Learn More About Performance Improvements in JDK 8 
• Fusion Middleware Tuning Performance of Oracle WebLogic Server (12.2.1.3.0)

JDK 8: Thread Stack Size Tuning

When you upgrade JDK, you should re-examine all JVM options you have set in your Java applications.  For example, let's look at thread stack size tuning specifically.  As suggested in [1], it states:

• In most applications, 128k happens to be enough for the Java thread stack.

However, after setting that, we have run into the following fatal exception:

The stack size specified is too small, Specify at least 228k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

In this article, we will discuss thread stack size tuning in JDK 8 (i.e., HotSpot VM).

Default Thread Stack Size

When a new thread is launched, the Java virtual machine creates a new Java stack for the thread. As mentioned earlier, a Java stack stores a thread's state in discrete frames.^[3] The Java virtual machine only performs two operations directly on Java Stacks: it pushes and pops frames.

The default thread stack size varies with JVM, OS and environment variables. To find out what your default ThreadStackSize is on your platform, use:^[1]

java -XX:+PrintFlagsFinal -version

A typical value is 512k. It is generally larger for 64bit JVMs because references are 8 bytes rather than 4 bytes in size (but, you can compress oops or class pointers if you choose).^[2] For example,^[4]

In Java SE 6, the default on Sparc is 512k in the 32-bit VM, and 1024k in the 64-bit VM. On x86 Solaris/Linux it is 320k in the 32-bit VM and 1024k in the 64-bit VM.

On Windows, the default thread stack size is read from the binary (java.exe). As of Java SE 6, this value is 320k in the 32-bit VM and 1024k in the 64-bit VM.

In JDK 8, every time the JVM creates a thread, the OS allocates some native memory to hold that thread’s stack, committing more memory to the process until the thread exits. Thread stacks are fully allocated (i.e., committed, not just reserved) when they are created.

This means that if your application spawns a lot of threads, this can consume a significant amount of memory which could otherwise be used by your application or OS (or it can eventually leads to OutOfMemoryError).

You can reduce your stack size by running with the -Xss option. For example:

java -server -Xss256k

or

java -server -XX:ThreadStackSize=256 

Note that if you have installed a 64-bit VM binary for Linux, you can omit -server option.^[5]

Virtual Memory Map

In JDK 8, HotSpot installation comes with a feature named Native Memory Tracking (default: disabled).  To enable it, use:

-XX:NativeMemoryTracking=[off|detail|summary]

After enabling NMT, you can examine the memory footprint taken by either Thread or Thread Stack using:

jcmd <pid> VM.native_memory [summary | detail | baseline | summary.diff | detail.diff | shutdown] [scale= KB | MB | GB]

 For example, on a 64-bit Linux platform, here is the thread stack size before and after setting -Xss256k:

Before

 Virtual memory map:

[0x0000000040049000 - 0x000000004014a000] reserved and committed 1028KB for Thread Stack from
    [0x00002aec741ca5e4] JavaThread::run()+0x24
    [0x00002aec74083268] java_start(Thread*)+0x108

After

Virtual memory map:

[0x0000000040078000 - 0x00000000400b9000] reserved and committed 260KB for Thread Stack from
    [0x00002b02c69156e4] JavaThread::run()+0x24
    [0x00002b02c67ce338] java_start(Thread*)+0x108

Conclusions

The thread stack is used to push stacks frames in nested method calls. If the nesting is so deep that the thread runs out of space, the thread dies with a StackOverflowError.^[8] If your applications use lots of recursive algorithms or if your applications are built on top of a framework utilizing MVC design pattern such as Oracle ADF, you may want to leave StackThreadSize as defaults.

However, thread stacks are quite large, particularly for a 64-bit JVM.  In [9], Scott Oaks has advised:

• As a general rule, many applications can actually run with a 128 KB stack size in a 32-bit JVM, and a 256 KB stack size in a 64-bit JVM.
• In a 64-bit JVM, there is usually no reason to set this value unless the machine is quite strained for physical memory and the smaller stack size will prevent applications from running out of native memory. 
• On the other hand, using a smaller (e.g., 128 KB) stack size on a 32-bit JVM is often a good idea, as it frees up memory in the process size and allows the JVM to utilize a larger heap.

Finally, the total footprint of the JVM has a significant effect on its performance. So, footprint is one aspect of Java performance that should be commonly monitored.

References

1. What Are the Default HotSpot JVM Values? 
2. JDK 8: UseCompressedClassPointers vs. UseCompressedOops
3. The Java Stack 
4. Frequently Asked Questions About the Java HotSpot VM 
5. HotSpot VM Binaries: 32-Bit vs. 64-Bit 
6. How to Debug Native OutOfMemory in JRockit 
7. HotSpot: Monitoring and Tuning Metaspace in JDK 8
8. Using -Xss to adjust Java default thread stack size to save memory and prevent StackOverflowError
9. Java Performance: The Definitive Guide (Strongly recommended)
10. Java thread stack size on 64-bit linux 
11. Learn More About Performance Improvements in JDK 8 
12. HotSpot Virtual Machine Garbage Collection Tuning Guide
13. Getting Started with the G1 Garbage Collector
14. Other JDK 8 articles on Xml and More
15. Tuning that was great in old JRockit versions might not be so good anymore
    
• Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
• Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).
16. HotSpot VM options (JDK 8)

G1 GC: Tuning Mixed Garbage Collections in JDK 8

To tune G1 GC (Garbage First Garbage Collector),  a place to start with is [1]. For the latest information, read [2].

Assuming you have minimal knowledge of G1 GC, the focus of this article is to tune the Mixed GC for better performance.

The Need to Tune Mixed GC

G1 GC is a generational garbage collector—read [3] to learn what eden, survivor, and old generation spaces are.  It also uses region-based architecture which divides large contiguous Java heap space into multiple fixed-sized heap regions.

The destination region for a particular object depends upon the object's age; an object that has aged sufficiently is evacuated to an old generation region (or promoted); otherwise, the object is evacuated to a survivor region and will be included in the CSet^[5] of the next young or mixed garbage collection.

During young collections, G1 GC adjusts its young generation (eden and survivor sizes) to meet its pause target. During mixed collections, the G1 GC adjusts the number of old gen regions that are collected based on a target number of mixed garbage collections, the percentage of live objects in each region of the heap, and the overall acceptable heap waste percentage (see details later).

Depending on your application's workload, Full GC could be expensive in G1 GC.  Since Mixed GC collects both young and old regions, you can get better performance by tuning Mixed GC—the goal is to reduce the number of Full GC's when your applications run.

More on Mixed Garbage Collections

Upon successful completion of a concurrent marking cycle, the G1 GC switches from performing young garbage collections to performing mixed garbage collections. In a mixed garbage collection, the G1 GC optionally adds some old regions to the set of eden and survivor regions that will be collected. The exact number of old regions added is controlled by a number of flags that will be discussed later. After the G1 GC collects a sufficient number of old regions (over multiple mixed garbage collections), G1 reverts to performing young garbage collections until the next marking cycle completes.

To summarize, Mixed GC can be characterized by:

• Collecting both young and old regions
• Denoted by: [GC pause (G1 Evacuation Pause) (mixed) in the GC log file^[2]
• Only after a completed marking cycle
• A sequence of mixed GC events to collect old regions, up to 8 by default

Tuning Mixed Garbage Collections

You can play with the following options to tune Mixed GC:^[1,2,4]

• -XX:InitiatingHeapOccupancyPercent 
• Percentage of the (entire) heap occupancy to start a concurrent GC cycle. It is used by GCs that trigger a concurrent GC cycle based on the occupancy of the entire heap, not just one of the generations.  The default value is 45.
• This option can be used to change the marking threshold
• If threshold is exceeded, a concurrent marking will be initiated next.
• The higher the threshold is, the less concurrent marking cycles will be, which also means the less mixed GC evacuation will be.
• -XX:G1MixedGCLiveThresholdPercent 
• This option can be used to change the threshold which determines whether a region should be added to the CSet or not.
• Only regions whose live data percentage are less than the threshold will be added to the CSet.
• The higher the threshold (default: 65) is, the more likely a region will be added to the CSet, which also means more mixed GC evacuation and longer evacuation time will happen. 
• -XX:G1HeapWastePercent
• Amount of reclaimable space, expressed as a percentage of the heap size that G1 will stop doing mixed GC's. If the amount of space that can be reclaimed from old generation regions compared to the total heap is less than this, G1 will stop mixed GC's.
• Current default is 10%.  A lower value, say 5%, will potentially cause G1 to add more expensive region(s) to evacuate for space reclamation.
• G1 will continue triggering mixed GC if the reclaimable is higher than the waste threshold.  So, there will be more mixed GC and maybe more expensive if you set this waste threshold lower.

Besides the above-mentioned options, you may also want to tune the following ones:

• -XX:G1MixedGCCountTarget 
• Sets the target number of mixed garbage collections after a marking cycle to collect old regions with at most G1MixedGCLIveThresholdPercent live data. 
• The default is 8 mixed garbage collections. The goal for mixed collections is to be within this target number.
• -XX:G1OldCSetRegionThresholdPercent
• Sets an upper limit on the number of old regions to be collected during a mixed garbage collection cycle. The default is 10 percent of the Java heap.
• -XX:G1HeapRegionSize
• Sets the size of a G1 region. The value will be a power of two and can range from 1MB to 32MB. 
• The goal is to have around 2048 regions based on the minimum Java heap size.

Conclusion

Each application is unique, you may need to tune G1 GC in an iterative process.  Note that all of the above options are product options except G1MixedGCLiveThresholdPercent and G1OldCSetRegionThresholdPercent, which are experimental options.  Be warned that: for the experimental options, Oracle may remove them at its discretion in the future releases.  

Acknowledgement

Some writings here are based on the feedback from Thomas Schatzl and Yu Zhang. However, the author would assume the full responsibility for the content himself.

References

1. Garbage First Garbage Collector Tuning
2. g1gc logs - Ergonomics -how to print and how to understand 
3. Understanding Garbage Collection  
4. Garbage First (G1) Garbage Collection Options
5. CSet
• The G1 GC reduces heap fragmentation by incremental parallel copying of live objects from one or more sets of regions (called Collection Set (CSet)) into different new region(s) to achieve compaction.
• The goal of G1 GC is to reclaim as much heap space as possible, starting with those regions that contain the most reclaimable space (i..e, garbage first), while attempting to not exceed the pause time goal.
6. G1 GC Glossary of Terms
7. Learn More About Performance Improvements in JDK 8 
8. HotSpot Virtual Machine Garbage Collection Tuning Guide
9. Garbage-First Garbage Collector (JDK 8 HotSpot Virtual Machine Garbage Collection Tuning Guide)
10. G1 GC: Humongous Objects and Humongous Allocations (Xml and More)
11. Other JDK 8 articles on Xml and More
12. Concurrent Marking in G1

JDK 8: Revisiting ReservedCodeCacheSize and CompileThreshold

In[1], someone has commented that:

Did you specify any extra JVM parameters to reach the state of full CodeCache? Some comments on the internet indicate this happens if you specify too low "-XX:CompileThreshold" and too much bytecode gets compiled by HotSpot very early.

when the following warning was seen:

VM warning: CodeCache is full. Compiler has been disabled.

In this article, we will look at tuning CompileThreshold and ReservedCodeCacheSize in JDK 8.

CompileThreshold

By default, CompileThreshold is set to be 10,000:

     intx CompileThreshold     = 10000       {pd product}

As described in [2], we know {pd product} means "platform-dependent product option".  Our platfrom is linux-x64 and that will be used for this discussion.

Very often, you see people setting the threshold lower.  For example

-XX:CompileThreshold=8000

Why?  Since the JIT compiler does not have time to compile every single method in an application, all code starts out initially running in the interpreter, and once it becomes hot enough it gets scheduled for compilation. To help determine when to convert bytecodes to compiled code, every method has two counters:

• Invocation counter
• Which is incremented every time a method is entered
• Backedge counter
•  Which is incremented every time control flow moves from a higher bytecode index to a lower one

Whenever either counter is incremented by the interpreter it checks them against a threshold, and if they cross this threshold, the interpreter requests a compile of that method.

The threshold used for the invocation counter is called the CompileThreshold, the backedge counter uses a more complex formula derived from CompileThreshold and OnStackReplacePercentage.  So, if you set the threshold lower, HotSpot compiles methods earlier.  And, in some cases, that can help the performance of server codes.

ReservedCodeCacheSize

A code cache is where JVM uses to store the native code generated for compiled methods.  As described in [3], to improve an application's performance, you can set the "reserved" code cache size:

• -XX:ReservedCodeCacheSize=256m

when tiered compilation is enabled for the HotSpot.  Basically it sets the maximum size for the compiler's code cache.  In [4], we have shown that an application can run faster if tiered compilation is enabled in a server environment.  However, code cashe size also needs to be specified larger.

What's New in JDK 8?

We have seen people setting the following JVM options:

• -XX:ReservedCodeCacheSize=256m -XX:+TieredCompilation

or

• -XX:CompileThreshold=8000 

in JDK 7.  In JDK 8, do we still need to set them?  The answer is that it depends on the platform.  On linux-x64 platforms, those setting are no longer necessary.  Here we will describe why.

In JDK 8, it chooses the following default values for linux-x64 platforms:

    bool TieredCompilation        = true       {pd product}     
    intx CompileThreshold         = 10000      {pd product}
    uintx ReservedCodeCacheSize   = 251658240  {pd product}

When tiered compilation is enabled, two things happen:

1. CompileThreshold is ignored
2. A bigger code cache is needed.  Internally, HotSpot will set it to be 240 MB (i.e., 48 MB * 5)

That's why we say that people don't need to set the following options anymore in JDK8:

• -XX:ReservedCodeCacheSize=256m -XX:+TieredCompilation 

or

• -XX:CompileThreshold=8000

Noted that “reserved” code cache is just an address space reservation, it does not really consume any additional physical memory unless it’s used.  On 64-bit platforms, it doesn’t hurt at all to set a higher value.  However, if you have set cache size to be too small, you will definitively see the negative impact on your application's performance.

Acknowledgement

Some writings here are based on the feedback from Igor Veresov and Vladimir Kozlov. However, the author would assume the full responsibility for the content himself.

References

1. VM warning: CodeCache is full. Compiler has been disabled.
2. HotSpot: What Does {pd product} Mean?  (Xml and More)
3. Performance Tuning with Hotspot VM Option: -XX:+TieredCompilation (Xml and More) 
4. A Case Study of Using Tiered Compilation in HotSpot  (Xml and More)
5. Useful JVM Flags – Part 4 (Heap Tuning)
6. g1gc logs - Ergonomics -how to print and how to understand 
7. G1 GC Glossary of Terms
8. Learn More About Performance Improvements in JDK 8 
9. HotSpot Virtual Machine Garbage Collection Tuning Guide
10. Other JDK 8 articles on Xml and More
11. Tuning that was great in old JRockit versions might not be so good anymore
    
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

JDK 8: UseCompressedClassPointers vs. UseCompressedOops

A new JVM option was introduced into JDK 8 after PermGen removal:

UseCompressedClassPointers

In this article, we will discuss the difference between  UseCompressedOops and UseCompressedClassPointers.

Default Values

As described in [2], you can find out the default values of UseCompressedClassPointers and UseCompressedOops:

     bool UseCompressedClassPointers           := true   {lp64_product}
     bool UseCompressedOops                    := true   {lp64_product}

Our platform is linux-x64 and these options are both set to be true based on ergonomics.

UseCompressedOops vs. UseCompressedClassPointers

CompressedOops are for the compression of pointers to objects in the Java Heap.  Class data is no
longer in the Java Heap and the compression of pointers to class data is done under the flag
UseCompressedClassPointers.  In next sections, we will discuss them in more details.

Oops and Compressed Oops

Oops are "ordinary" object pointers.  Specifically, a pointer into the GC-managed heap. Implemented as a native machine address, not a handle. Oops may be directly manipulated by compiled or interpreted Java code, because the GC knows about the liveness and location of Oops within such code.  Oops can also be directly manipulated by short spans of C/C++ code, but must be kept by such code within handles across every safepoint.

Compressed Oops represent managed pointers (in many but not all places in the JVM) as 32-bit values which must be scaled by a factor of 8 and added to a 64-bit base address to find the object they refer to in Java Heap.

Compressed Class Pointers

Objects (in its 2nd word) have a pointer  to VM Metadata class, which can be compressed.  If compressed, it uses a base which points to the Compressed Class Pointer Space.

Before we continue, you need to know what Metaspace and Compressed Class Pointer Space  are. A Compressed Class Pointer Space (which is logically part of Metaspace) is introduced for 64 bit platforms. Whereas the Compressed Class Pointer Space contains only class metadata, the Metaspace can contain all other large class metadata including methods, bytecode etc.

For 64 bit platforms, the default behavior is using compressed (32 bit) object pointers (-XX:+UseCompressedOops) and compressed (32 bit) class pointers (-XX:+UseCompressedClassPointers).  However, you can modify default settings if you like.  When you do  modify them, be warned that there is a dependency between the two options—i.e., UseCompressedOops must be on for UseCompressedClassPointers to be on.

To summarize it, the differences between Metaspace and Compressed Class Pointer Space are :[3]

• Compressed Class Pointer Space contains only class metadata
• InstanceKlass, ArrayKlass
• Only when UseCompressedClassPointers true
• These include Java virtual tables for performance reasons
• Metaspace contains all other class metadata that can be large.
• Methods, Bytecodes, ConstantPool ...

 

References

1. HotSpot: Monitoring and Tuning Metaspace in JDK 8 (Xml and More)
2. What Are the Default HotSpot JVM Values?
3. Metaspace in Java 8 (good)  
4. HotSpot Glossary of Terms 
5. VM Class Loading
6. Learn More About Performance Improvements in JDK 8 
7. Garbage-First Garbage Collector Tuning
8. Understanding Compressed References (JRockit)
• Similar to HotSpot's CompressedOops, Jrockit uses -XXcompressedRefs command option.

HotSpot: Sizing System Dictionary in JDK 8

The system dictionary is an internal JVM data structure which holds all the classes loaded by the system.  As described in JDK-7114376,^[1]

The System Dictionary hashtable bucket array size is fixed at 1009. This value is too small for large programs with many names, too large for small programs and just about right for medium sized ones. The default should remain at 1009, but it should be possible to override it on the command line.

Finally, this has happened in JDK 8 and you can set hashtable bucket array size to be larger if you have more classes loaded in your applications by using:^[2]

• -XX:+UnlockExperimentalVMOptions  -XX:PredictedLoadedClassCount=<#>

This can help calls like Class.forName(), which do lookups into this data structure.

In this article, we will look into sizing system dictionary in more details.

Performance of System Dictionary

In 1.4.2, there were some performance enhancements.  One of them is making system dictionary reads lock-free.^[3] From this enhancement, you probably can guess that tuning system dictionary could be important to the JVM's performance.

The current number of buckets in hash table for system dictionary is set to be 1009 by default, which is relatively small for an application which has 49987 classes loaded as shown below:

Loaded   Bytes Unloaded   Bytes       Time
49987 110488.8     2453  7743.8     105.24

From the experience, we know:^[5]

In a good hash table, each bucket has zero or one entries, and sometimes two or three, but rarely more than that.

 Assuming the average ideal length of buckets is three, the hashtable bucket array size then should be:

Ideal hashtable bucket array size = 49987 / 3 = 16662

 

System Dictionary Tuning

Seeing 49987classes loaded in our application at end of its run, we decided to set:

-XX:PredictedLoadedClassCount=16661 (a prime)

Note that PredictedLoadedClassCount is an experimental flag.  So, you also need to set:

-XX:+UnlockExperimentalVMOptions

before it in the command line.

When the JVM allocates memory, the largest chunk is the java heap. The system dictionary is in C heap and it is the loaded class cache. The goal of setting PredictedLoadedClassCount flag is to increase the size of the system dictionary in order to make lookups of loaded classes faster. Before every class being loaded, it requires a checking to see if the class is already loaded.  Larger system dictionary will improve JVM performance during this class loading and resolution phase.

Note that the total number of entries in the hashtable does not change— that is based on the number of loaded classes. What sizing of system dictionary do is to increase the spread of the entries so that the average length of the buckets under a single hash result could be reduced. This would reduce the time it takes to find a given entry.

How to Find Number of Loaded Classes?

There are multiple ways to find out number of loaded classes in an application.  For example, you can use -Xverbose:class to determine what classes are loaded.   Another way to find out is what we have done in this article by setting -XX:+PerfDataSaveToFile and then use "jstat -class" command to decipher the class statistics offline:

$<snipped>/jdk-hs/bin/jstat -class file:////slot/myserver/appmgr/APPTOP/instance/domains/slcaf977.us.oracle.com/CRMDomain/hsperfdata_9872

Loaded   Bytes Unloaded   Bytes       Time
49987 110488.8     2453  7743.8     105.24

Acknowledgement

Some writings here are based on the feedback from Mikael Gerdin and Karen Kinnear. However, the author would assume the full responsibility for the content himself.

References

1. Make system dictionary hashtable bucket array size configurable  (JDK-7114376)
2. Tuning the JVM (at 43:02 mark)
3. Java 2 Platform, Standard Edition (J2SE Platform), version 1.4.2 Performance White Paper 
4. The Class Loader Subsystem 
5. Hash table (Wikipedia) 
6. VM Class Loading
• Actually, the HotSpot VM maintains three main hash tables to track class loading
• SystemDictionary  
• PlaceholderTable 
• LoaderConstraintTable
7. Garbage-First Garbage Collector Tuning

jstat Tool: New Metaspace Statistics from -gc Option

In [1], we have shown one way of estimating the size of metaspace^[2] using the following command:

jstat -class

However, there are other metadata or implementation overhead (i.e., MetaBlock overhead) not accounted for in the reported sizes.

In this article, we will introduce another way of monitoring the sizes needed for "Metaspace" and "Class Space".

Heap Statistics Reported by PrintGC

As described in [2], the following heap statistics would be printed at the exit of JVM if you have enabled PringGC (i.e., PrintGCDetails or PrintHeapAtGC):

Heap
 garbage-first heap   total 2097152K, used 1680819K [0x0000000080000000, 0x0000000100000000, 0x0000000100000000)
  region size 4096K, 27 young (110592K), 9 survivors (36864K)
 Metaspace       used 327768K, capacity 340303K, committed 340780K, reserved 1349632K
  class space    used 37537K, capacity 40670K, committed 40832K, reserved 1048576K

jstat -gc Command

jstat -gc can be used to display Garbage-collected heap statistics.  In JDK 8, it would display Metaspace's statistics, but not PermGen's.

-bash-3.2$ jstat -gc 18990 1000 3
Warning: Unresolved Symbol: sun.gc.generation.2.space.0.capacity substituted NaN
Warning: Unresolved Symbol: sun.gc.generation.2.space.0.used substituted NaN
 S0C   S1C      S0U   S1U     EC       EU       OC         OU           PC     PU  
 0.0   16384.0  0.0   16384.0 94208.0  12288.0  1986560.0  1783935.8    �      �     
 0.0   16384.0  0.0   16384.0 94208.0  57344.0  1986560.0  1775743.8    �      �     
 0.0   16384.0  0.0   16384.0 94208.0  69632.0  1986560.0  1775743.8    �      �      

At the first try, it has printed the above three lines.  However, the default jstat was run, which belongs to JDK 7.  So, you still see headers such as "PC" and "PU".  Also, you could see some gibberish text because of "unresolved symbols."  To run jstat, you need to use the correct version (i.e., from your JDK 8 installation).  After correcting this error, we have seen: 

-bash-3.2$ cd JVMs/
-bash-3.2$ jdk-hs/bin/jstat  -gc 18990 1000 3
 S0C   S1C      S0U   S1U     EC       EU       OC         OU         MC       MU       CCSC    CCSU  
 0.0   28672.0  0.0   28672.0 163840.0 45056.0  1904640.0  1663244.2  340992.0 327753.1 40960.0 37549.0  
 0.0   28672.0  0.0   28672.0 163840.0 77824.0  1904640.0  1663244.2  340992.0 327753.1 40960.0 37549.0
 0.0   28672.0  0.0   28672.0 163840.0 86016.0  1904640.0  1663244.2  340992.0 327753.1 40960.0 37549.0 


Note that we now see "MC", "MU", "CCSC", and "CCSU" instead of "PC" and "PU".

Meaning of "MC", "MU", "CCSC", and "CCSU"

There is no good documentation on what they mean yet.  But, a good guess would be:

• MC (Metaspace Committed)
• MU (Metaspace Used)
• CCSC (Compressed Class Space Committed)
• CCSU (Compressed Class Space Used)

If you check the values printed out from "jstat -gc" and the values printed out at JVM's exit time.  They have similar values.  Note that the "jstat -gc" command was issued close to the end of our JVM run.

To learn more about these new headers, read [2].

Acknowledgement

Some writings here are based on the feedback from Jon Masamitsu. However, the author would assume the full responsibility for the content himself.

References

1. HotSpot: Monitoring and Tuning Metaspace in JDK 8
2. HotSpot: Understanding Metaspace in JDK 8
3. jstat - Java Virtual Machine Statistics Monitoring Tool
4. Interpreting jstat results
5. JavaOne Wednesday: Permanent Generation Removal Overview
6. HotSpot: A Case Study of MetaspaceSize Tuning in JDK 8

HotSpot: Understanding Metaspace in JDK 8

In JDK 8, it will print out the following information when JVM exits:

Heap

 <snipped> 

 Metaspace       used 2425K, capacity 4498K, committed 4864K, reserved 1056768K
  class space    used 262K, capacity 386K, committed 512K, reserved 1048576K

if you have enabled GC printing using:

-XX:+PrintGCDetails -XX:+PrintGCTimeStamps

Note that if PrintGCDetails is enabled, PrintGC^[1] is also enabled.

In this article, we will examine what the above lines mean.

Metaspace

Internal JVM memory management is, to a large extent, kept off the Java heap and allocated natively in the operating system, through system calls like malloc or mmap. This non-heap system memory allocated by the JVM is referred to as native memory. Similar to the Oracle JRockit and IBM JVM's, the JDK 8 HotSpot JVM is now using native memory for the representation of class metadata, which is called Metaspace.^[2,4]

In JDK 8, Hotspot explicitly manages the space used for metadata. Space is requested from the OS and then divided into chunks. A class loader will allocate space for metadata from its chunks (a chunk is to bound to a specific class loader). When classes are unloaded for a class loader, its chunks are recycled for reuse or returned to the OS.   Note that metadata uses mmap'ed space and not malloc'ed space.

Used/Capacity/Committed/Reserved

When JVM exits, it prints the following information if you have enabled GC printing:

 Metaspace       used 2425K, capacity 4498K, committed 4864K, reserved 1056768K
  class space    used 262K, capacity 386K, committed 512K, reserved 1048576K

In the “Metaspace” line, the “used” is the amount of space used for loaded classes and other metadata. The “capacity” is the space available for metadata in currently allocated chunks. The “committed” value is the amount of space available for chunks. The “reserved” is the amount of space reserved (but not necessarily committed) for metadata. On the “class space” line, those values are the corresponding values for the class area in Metaspace when compressed class pointers are used.

Mataspace is dynamically managed by HotSpot.  Metadata are deallocated when their class loader are garbage collected (GC'ed).  A high water mark is used for inducing a GC—when committed memory of all metaspaces reaches this level, a GC is triggered.  You can use the following flag:

-XX:MetaspaceSize=<size>

to specify the initial high water level. 

The interaction between metaspace growth and GC can be summarized as follows:

Metaspaces expand the native memory it is using until it gets to some level (starts at MetaspaceSize). When it hits that level, it does a GC to see if classes can be unloaded. After the GC, it can use freed space in it for metadata. If not enough space has been freed, it uses more native memory.

After the GC it also decides what the next level is for doing a GC to unload classes.  The level mostly increases to have fewer GC's. It sometimes will decrease if lots of space has been freed due to class unloading.  If MetaspaceSize is set higher, then fewer GC will be done early.

Acknowledgement

Some writings here are based on the feedback from Jon Masamitsu. However, the author would assume the full responsibility for the content himself.

References

1. Java HotSpot VM Options
2. HotSpot: Monitoring and Tuning Metaspace in JDK 8
3. Latest JDK 8 and JDK 7 available here
4. JavaOne Wednesday: Permanent Generation Removal Overview   
5. jstat Tool: New Metaspace Statistics from -gc Option 
6. Metatspace in JDK 8 (good) 
7. VM Class Loading
8. Learn More About Performance Improvements in JDK 8 

HotSpot: Monitoring and Tuning Metaspace in JDK 8

If you upgrade from JDK 7 to JDK 8, there are some changes in JVM that you should pay attention to.  One of them is:

• The removal of Permanent Generation (PermGen) space.^[1]  

Similar to the Oracle JRockit and IBM JVM's, the JDK 8 HotSpot JVM is now using native memory for the representation of class metadata, which is called Metaspace.  This may have removed the old OOM Error.^[2] But ...

As described in [3], proper monitoring and tuning of the Metaspace is still required in order to limit the frequency or delay of metaspace garbage collections.  In this article, we will show how to achieve that.

Tuning Metaspace

When space (either PermGen or Metaspace) is filled up, class need to be unloaded.  In the PermGen era, when PermGen is filled up, a Garbage Collection (or GC) would occur and unload classes.  Without PermGen, we still need to have some way to know when to do a GC to unload classes.  This is when

-XX:MetaspaceSize=<size>

comes into play.   When the amount of space used for classes reaching MetaspaceSize, a GC is done to see if there are classes to be unloaded.  If you know that you are going to have lots of classes loaded and use more than 20M (this is the default value for my platform) class data, increasing MetaspaceSize can delay the GC's that are done to check for class unloading.  During your upgrade from JDK 7 to 8, as a first estimate, you can use the old value of PermSize to be your new MetaspaceSize.  You can also read [4] for more details.

Since Metaspace uses native memory, class metadata allocation theoretically is limited by amount of available native memory (capacity will of course depend if you use a 32-bit JVM vs. 64-bit along with OS virtual memory availability).  However, if you want to limit the amount of native memory used for class metadata, you can set:

-XX:MaxMetaspaceSize=<size>

which can restrict the extension of metaspace to the maximum at runtime.

How Many Classes Are Loaded?

There are multiple way^[6,7] to find out how many classes are loaded and how much space they would take.  Here we introduce one way of finding that information.

jps is a Java tool that  you can use to  list the instrumented HotSpot Java Virtual Machines (JVMs) on the target system.  For example, here are the list of JVMs that are running on our Linux.

$ jps -l

32520 weblogic.Server
8936 weblogic.Server
11598 weblogic.Server
32646 weblogic.Server
7591 weblogic.Server
27429 weblogic.Server

To gather class information, you can use jstat tool which displays performance statistics for an instrumented HotSpot Java virtual machine (JVM).  For example, jstat has attached to process 27429 and displayed the statistics on the behavior of the class loader as below:

$ jstat -class 27429 1000 6

Loaded  Bytes  Unloaded  Bytes     Time
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22
 46167 98340.6      406  1494.6      54.22

Class Loader Statistics

Column	Description
Loaded	Number of classes loaded.
Bytes	Number of Kbytes loaded.
Unloaded	Number of classes unloaded.
Bytes	Number of Kbytes unloaded.
Time	Time spent performing class load and unload operations.

  

From the above output, we know that 46167 classes were loaded and they have taken 96M.  At the time of snapshots, HotSpot has also unloaded 406 classes from which 1.46 M were freed.  Based on this information, you can then decide how to set your MetaspaceSize and MaxMetaspaceSize (not advised) as described above.  However, be warned that once you have set MaxMetaspaceSize, you may still run into OOM error if metaspace cannot be extended further.

Acknowledgement

Some writings here are based on the feedback from Jon Masamitsu. However, the author would assume the full responsibility for the content himself.

References

1. Java 7 features - PermGen removal
2. 64-bit java.lang.OutOfMemoryError: PermGen space
3. Java 8: From PermGen to Metaspace
4. HotSpot: Understanding Metaspace in JDK 8 
5. Latest JDK 8 and JDK 7 available here
6. JavaOne Wednesday: Permanent Generation Removal Overview  
7. jstat Tool: New Metaspace Statistics from -gc Option 
8. VM Class Loading
9. Other JDK 8 articles on Xml and More
10. Tuning that was great in old JRockit versions might not be so good anymore
    
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).