JDK 8: When Softly Referenced Objects Are Flushed?

There are four different degrees of reference strength in Java language:[1]

1. Strong
• StringBuffer strongRef = new StringBuffer();
2. Soft
• SoftReference softRef = new SoftReference(obj);
3. Weak
• WeakReference weakRef = new WeakReference(widget);
4. Phantom
• PhantomReference phantomRef =new PhantomReference(bool,rq);

in order from strongest to weakest.

In this article, we will focus on softly referenced objects and when they will be flushed by Garbage Collector (GC) in HotSpot.

SoftReference vs WeakReference

A soft reference is exactly like a weak reference, except that it is less eager to throw away the object to which it refers. An object which is only weakly reachable (the strongest references to it are WeakReferences) will be discarded at the next garbage collection cycle, but an object which is softly reachable will generally stick around for a while.

SoftReferences aren't required to behave any differently than WeakReferences, but in practice softly reachable objects are generally retained as long as memory is in plentiful supply. This makes them an excellent foundation for a cache since you can let the garbage collector worry about both how reachable the objects are and how badly it needs the memory they are consuming.

When Softly Referenced Objects Are Flushed?

Prior to version 1.3.1, the Java HotSpot VMs cleared soft references whenever it found them. Starting with 1.3.1, softly reachable objects will remain alive for some amount of time after the last time they were referenced. The default value is one second of lifetime per free megabyte in the heap. This value can be adjusted using the -XX:SoftRefLRUPolicyMSPerMB flag, which accepts integer values representing milliseconds. For example, to change the value from one second to 2.5 seconds, use this flag:

-XX:SoftRefLRUPolicyMSPerMB=2500

When, exactly, is a soft reference freed?  Here are the factors to be considered:

• First the referent must not be strongly referenced elsewhere
• If the soft reference is the only remaining reference to its referent, the referent is freed during the next GC cycle only if the time it was last accessed is greater than (AmountOfFreeMemoryInMB * SoftRefLRUPolicyMSPerMB).
• AmountOfFreeMemoryInMB is calculated differently for server and client VMs:
• • • Server VM
    • Uses the maximum possible heap size (as set with the -Xmx option) to calculate free space remaining
    • -Xmx therefore has a significant effect on when soft references are garbage collected.
    • The general tendency is for the Server VM to grow the heap rather than flush soft references
    • Client VM
    • Uses the current heap size to calculate the free space
    • The Client VM will have a greater tendency to flush soft references rather than grow the heap

Conclusion

• When to use SoftReference?
• Soft references are used when the object in question has a good chance of being reused in the future,
• For example, as a cache implementation
• Soft references are essentially one large, least-recently-used cache
• Performance consideration
• The key to getting good performance from that cache is to make sure that it is cleared on a timely basis.
• Eagerly vs. lazily reclaimation of softly referenced objects
• You can tune how eagerly they are reclaimed by setting:
• -XX:SoftRefLRUPolicyMSPerMB flag
• Default is 1000 ms. Optimal setting depends on the applications and which GC is used.
• This behavior is not part of the VM specification, however, and is subject to change in future releases. Likewise the -XX:SoftRefLRUPolicyMSPerMB flag is not guaranteed to be present in any given release.

References

1. Understanding Weak References Blog (good)
2. Frequently Asked Questions About the Java HotSpot VM
3. HotSpot Virtual Machine Garbage Collection Tuning Guide
4. Garbage-First Garbage Collector Tuning
5. Other JDK 8 articles on Xml and More
6. Tuning that was great in old JRockit versions might not be so good anymore
   
   • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
   • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

JDK 8: Is Tuning MaxNewSize in G1 GC a Good Idea?

For a throughput GC such as Parallel GC, it often helps performance by setting MaxNewSize (or -Xmn).  For G1 GC,^[1] is it the same? The short answer is yes/no and it depends (confession: I said "no" because I haven't done enough tests to exclude it).

In this article, we will demonstrate one case that setting MaxNewSize for G1 actually hurts the application's performance.  BTW, after benchmarking G1 in JDK 7, another author has claimed that:^[14]

• There is little to gain and much to lose from setting the new generation size explicitly.

Live Data Size

To tune the heap size of a Java Application, it's important to find out what its live data size is.  To learn how to estimate a Java application's live data size, read [2].  For the benchmark we are using, its live data size is around 1400 MB. With this benchmark, we are interested in knowing how G1 performs with MaxNewSize set to 400MB and Java heap set to 2GB.

Regularly we use the following settings for the G1 tuning:

• -Xms2g -Xmx2g -XX:+UseG1GC

plus some options for mixed GC (read [3]). So, for this experiment, we have added MaxNewSize setting like this:

• -Xms2g -Xmx2g -XX:+UseG1GC -Xmn400m

This has turned out hurting the performance a lot. Note that -Xmn400m is the optimal setting of our benchmark if Parallel GC is used.

Concurrent Cycle in G1 GC

G1 has four main operations:^[5]

• A young collection
• A background, concurrent cycle
• A mixed collection
• If necessary, a full GC

Without much ado, read [5-9] for the needed background of G1 GC.

G1 is a concurrent collector.  Its concurrent cycle has multiple phases—some of which stop all application threads (denoted by STW) and some of which do not:

• Initial-mark (STW)
• Denoted by: 
• [GC pause (G1 Evacuation Pause) (young) (initial-mark)
• [GC pause (Metadata GC Threshold) (young) (initial-mark)
• The G1 GC marks the roots during this phase. This phase is piggybacked on a normal (STW) young garbage collection.
• Root Region Scan
• Denoted by: 
• [GC concurrent-root-region-scan-start]
• [GC concurrent-root-region-scan-end
• The G1 GC scans survivor regions of the initial mark for references to the old generation and marks the referenced objects. 
• This phase runs concurrently with the application (not STW) and must complete before the next STW young garbage collection can start.
• Concurrent marking
• Denoted by:
• [GC concurrent-mark-start]
• [GC concurrent-mark-end
• The G1 GC finds reachable (live) objects across the entire heap. 
• This phase happens concurrently with the application, and can be interrupted by STW young garbage collections.
• Remark (STW)
• Denoted by:
• [GC remark
• This phase is STW collection and helps the completion of the marking cycle. 
• Finds objects that were missed by the concurrent mark phase due to updates by Java application threads to objects after the concurrent collector had finished tracing that object.
• G1 GC drains SATB buffers, traces unvisited live objects, and performs reference processing.
• Cleanup (STW)
• Denoted by:
• [GC cleanup 1936M->1931M(2048M)
• Prepare for next concurrent collection by clearing data structures.
• In this final phase, the G1 GC performs the STW operations of accounting and RSet scrubbing. 
• During accounting, the G1 GC identifies completely free regions and mixed garbage collection candidates. The cleanup phase is partly concurrent when it resets and returns the empty regions to the free list.
• Concurrent cleanup:
• Denoted by:
• [GC concurrent-cleanup-start]
• [GC concurrent-cleanup-end
• In this phase, G1 reclaims regions which were found empty during marking.
• Adds empty regions to the free list—i.e., thread local free lists are merged into global free list

G1 GC uses the Snapshot-At-The-Beginning (SATB) algorithm, which takes a snapshot of the set of live objects in the heap at the start of a marking cycle.  During the concurrent cycle, young garbage collections are allowed, which are triggered when eden fills up (note that initial-mark is implemented using a young collection cycle) .

The set of live objects after marking cycle is composed of the live objects in the snapshot, and the objects allocated since the start of the marking cycle. The G1 GC marking algorithm uses a pre-write barrier to record and mark objects that are part of the logical snapshot.

After the concurrent cycle, we expect to see:^[5]

• The eden regions before the marking cycle have been completely freed and new eden regions have been  allocated
• Old regions could be more occupied because the promotion of live objects from young regions
• Some old regions are identified to be mostly garbage and become candidates in later mixed or old collection cycles

Diagnosis

The reason of setting -Xmn400m has caused the regression of G1 is:

The option has forced G1 to use a young generation space of up to 400 MB and leaves G1 just 200 MB (i.e. 2048MB - 400MB - 1400MB) breathing room for shuffling live objects around.    

The small free space has negative impact on G1's concurrent cycles, which has caused most marking cycles not being completed in time.  

Before setting -Xmn400m on the command line, we have found only one instance of aborted marking cycle, which is denoted by:

• [GC concurrent-mark-abort]

After setting it, we have seen marking cycle was aborted 105 times out of 150 instances.  The lesson we have learnt here is that for a tight heap like our benchmark, it's not a good idea to set -Xmn400m for G1.  Without setting MaxNewSize, G1's ergonomics will dynamically adjust young generation space and it turns out that G1 can do a better job in this case.

Differences between Parallel GC and G1 GC

Note that -Xmn400m is the optimal setting for our benchmark if Parallel GC is used.  When we set MaxNewSize to be 400MB for G1, our benchmark regressed.  So, what's the difference between G1 GC and Parallel GC?

Initially, G1 GC was designed to replace CMS ^[10] for its relatively lower and more predictable pause times.  This is different from the design of Parallel GC which aims for higher throughput.  To help gain higher throughput, Parallel GC tries to:

• adjust young generation as large as possible and let it run into a full GC
• In our 4-hour experiments, we usually see around 50 full GC's in Parallel GC
• However, for G1's performance tuning, we want to avoid full GC's (see [11] for how)
• Which means more frequent full GC is expected
• Note that Parallel GC's Full GC pause time is shorter than G1's in the current JDK 8 releases

To conclude, if you upgrade JDK or switch garbage collector,^[7] always re-evaluate original optimal VM settings.

Acknowledgement

Some writings here are based on the feedback from Thomas Schatzl. However, the author would assume the full responsibility for the content himself.

References

1. Garbage First Garbage Collector Tuning
2. JRockit: How to Estimate the Size of Live Data Set
3. g1gc logs - basic - how to print and how to understand
4. g1gc logs - Ergonomics -how to print and how to understand
5. Java Performance: The Definitive Guide (Strongly recommended)
6. Garbage First Garbage Collector Tuning - Oracle
7. Our Collectors by Jon Masamitsu
8. Understanding Garbage Collection
9. HotSpot VM Performance Tuning Tips
10. Understanding CMS GC Logs
11. G1 GC: Tuning Mixed Garbage Collections in JDK 8
12. G1 GC Glossary of Terms
13. Learn More About Performance Improvements in JDK 8 
14. Benchmarking G1 and other Java 7 Garbage Collectors
15. HotSpot Virtual Machine Garbage Collection Tuning Guide
16. Getting Started with the G1 Garbage Collector
17. Garbage-First Garbage Collector (JDK 8 HotSpot Virtual Machine Garbage Collection Tuning Guide)
18. Other JDK 8 articles on Xml and More
19. Tuning that was great in old JRockit versions might not be so good anymore
    
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

JDK 8: Revisiting ReservedCodeCacheSize and CompileThreshold

In[1], someone has commented that:

Did you specify any extra JVM parameters to reach the state of full CodeCache? Some comments on the internet indicate this happens if you specify too low "-XX:CompileThreshold" and too much bytecode gets compiled by HotSpot very early.

when the following warning was seen:

VM warning: CodeCache is full. Compiler has been disabled.

In this article, we will look at tuning CompileThreshold and ReservedCodeCacheSize in JDK 8.

CompileThreshold

By default, CompileThreshold is set to be 10,000:

     intx CompileThreshold     = 10000       {pd product}

As described in [2], we know {pd product} means "platform-dependent product option".  Our platfrom is linux-x64 and that will be used for this discussion.

Very often, you see people setting the threshold lower.  For example

-XX:CompileThreshold=8000

Why?  Since the JIT compiler does not have time to compile every single method in an application, all code starts out initially running in the interpreter, and once it becomes hot enough it gets scheduled for compilation. To help determine when to convert bytecodes to compiled code, every method has two counters:

• Invocation counter
• Which is incremented every time a method is entered
• Backedge counter
•  Which is incremented every time control flow moves from a higher bytecode index to a lower one

Whenever either counter is incremented by the interpreter it checks them against a threshold, and if they cross this threshold, the interpreter requests a compile of that method.

The threshold used for the invocation counter is called the CompileThreshold, the backedge counter uses a more complex formula derived from CompileThreshold and OnStackReplacePercentage.  So, if you set the threshold lower, HotSpot compiles methods earlier.  And, in some cases, that can help the performance of server codes.

ReservedCodeCacheSize

A code cache is where JVM uses to store the native code generated for compiled methods.  As described in [3], to improve an application's performance, you can set the "reserved" code cache size:

• -XX:ReservedCodeCacheSize=256m

when tiered compilation is enabled for the HotSpot.  Basically it sets the maximum size for the compiler's code cache.  In [4], we have shown that an application can run faster if tiered compilation is enabled in a server environment.  However, code cashe size also needs to be specified larger.

What's New in JDK 8?

We have seen people setting the following JVM options:

• -XX:ReservedCodeCacheSize=256m -XX:+TieredCompilation

or

• -XX:CompileThreshold=8000 

in JDK 7.  In JDK 8, do we still need to set them?  The answer is that it depends on the platform.  On linux-x64 platforms, those setting are no longer necessary.  Here we will describe why.

In JDK 8, it chooses the following default values for linux-x64 platforms:

    bool TieredCompilation        = true       {pd product}     
    intx CompileThreshold         = 10000      {pd product}
    uintx ReservedCodeCacheSize   = 251658240  {pd product}

When tiered compilation is enabled, two things happen:

1. CompileThreshold is ignored
2. A bigger code cache is needed.  Internally, HotSpot will set it to be 240 MB (i.e., 48 MB * 5)

That's why we say that people don't need to set the following options anymore in JDK8:

• -XX:ReservedCodeCacheSize=256m -XX:+TieredCompilation 

or

• -XX:CompileThreshold=8000

Noted that “reserved” code cache is just an address space reservation, it does not really consume any additional physical memory unless it’s used.  On 64-bit platforms, it doesn’t hurt at all to set a higher value.  However, if you have set cache size to be too small, you will definitively see the negative impact on your application's performance.

Acknowledgement

Some writings here are based on the feedback from Igor Veresov and Vladimir Kozlov. However, the author would assume the full responsibility for the content himself.

References

1. VM warning: CodeCache is full. Compiler has been disabled.
2. HotSpot: What Does {pd product} Mean?  (Xml and More)
3. Performance Tuning with Hotspot VM Option: -XX:+TieredCompilation (Xml and More) 
4. A Case Study of Using Tiered Compilation in HotSpot  (Xml and More)
5. Useful JVM Flags – Part 4 (Heap Tuning)
6. g1gc logs - Ergonomics -how to print and how to understand 
7. G1 GC Glossary of Terms
8. Learn More About Performance Improvements in JDK 8 
9. HotSpot Virtual Machine Garbage Collection Tuning Guide
10. Other JDK 8 articles on Xml and More
11. Tuning that was great in old JRockit versions might not be so good anymore
    
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

JDK 8: UseCompressedClassPointers vs. UseCompressedOops

A new JVM option was introduced into JDK 8 after PermGen removal:

UseCompressedClassPointers

In this article, we will discuss the difference between  UseCompressedOops and UseCompressedClassPointers.

Default Values

As described in [2], you can find out the default values of UseCompressedClassPointers and UseCompressedOops:

     bool UseCompressedClassPointers           := true   {lp64_product}
     bool UseCompressedOops                    := true   {lp64_product}

Our platform is linux-x64 and these options are both set to be true based on ergonomics.

UseCompressedOops vs. UseCompressedClassPointers

CompressedOops are for the compression of pointers to objects in the Java Heap.  Class data is no
longer in the Java Heap and the compression of pointers to class data is done under the flag
UseCompressedClassPointers.  In next sections, we will discuss them in more details.

Oops and Compressed Oops

Oops are "ordinary" object pointers.  Specifically, a pointer into the GC-managed heap. Implemented as a native machine address, not a handle. Oops may be directly manipulated by compiled or interpreted Java code, because the GC knows about the liveness and location of Oops within such code.  Oops can also be directly manipulated by short spans of C/C++ code, but must be kept by such code within handles across every safepoint.

Compressed Oops represent managed pointers (in many but not all places in the JVM) as 32-bit values which must be scaled by a factor of 8 and added to a 64-bit base address to find the object they refer to in Java Heap.

Compressed Class Pointers

Objects (in its 2nd word) have a pointer  to VM Metadata class, which can be compressed.  If compressed, it uses a base which points to the Compressed Class Pointer Space.

Before we continue, you need to know what Metaspace and Compressed Class Pointer Space  are. A Compressed Class Pointer Space (which is logically part of Metaspace) is introduced for 64 bit platforms. Whereas the Compressed Class Pointer Space contains only class metadata, the Metaspace can contain all other large class metadata including methods, bytecode etc.

For 64 bit platforms, the default behavior is using compressed (32 bit) object pointers (-XX:+UseCompressedOops) and compressed (32 bit) class pointers (-XX:+UseCompressedClassPointers).  However, you can modify default settings if you like.  When you do  modify them, be warned that there is a dependency between the two options—i.e., UseCompressedOops must be on for UseCompressedClassPointers to be on.

To summarize it, the differences between Metaspace and Compressed Class Pointer Space are :[3]

• Compressed Class Pointer Space contains only class metadata
• InstanceKlass, ArrayKlass
• Only when UseCompressedClassPointers true
• These include Java virtual tables for performance reasons
• Metaspace contains all other class metadata that can be large.
• Methods, Bytecodes, ConstantPool ...

 

References

1. HotSpot: Monitoring and Tuning Metaspace in JDK 8 (Xml and More)
2. What Are the Default HotSpot JVM Values?
3. Metaspace in Java 8 (good)  
4. HotSpot Glossary of Terms 
5. VM Class Loading
6. Learn More About Performance Improvements in JDK 8 
7. Garbage-First Garbage Collector Tuning
8. Understanding Compressed References (JRockit)
• Similar to HotSpot's CompressedOops, Jrockit uses -XXcompressedRefs command option.