JDK 8: Thread Stack Size Tuning

When you upgrade JDK, you should re-examine all JVM options you have set in your Java applications.  For example, let's look at thread stack size tuning specifically.  As suggested in [1], it states:

• In most applications, 128k happens to be enough for the Java thread stack.

However, after setting that, we have run into the following fatal exception:

The stack size specified is too small, Specify at least 228k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

In this article, we will discuss thread stack size tuning in JDK 8 (i.e., HotSpot VM).

Default Thread Stack Size

When a new thread is launched, the Java virtual machine creates a new Java stack for the thread. As mentioned earlier, a Java stack stores a thread's state in discrete frames.^[3] The Java virtual machine only performs two operations directly on Java Stacks: it pushes and pops frames.

The default thread stack size varies with JVM, OS and environment variables. To find out what your default ThreadStackSize is on your platform, use:^[1]

java -XX:+PrintFlagsFinal -version

A typical value is 512k. It is generally larger for 64bit JVMs because references are 8 bytes rather than 4 bytes in size (but, you can compress oops or class pointers if you choose).^[2] For example,^[4]

In Java SE 6, the default on Sparc is 512k in the 32-bit VM, and 1024k in the 64-bit VM. On x86 Solaris/Linux it is 320k in the 32-bit VM and 1024k in the 64-bit VM.

On Windows, the default thread stack size is read from the binary (java.exe). As of Java SE 6, this value is 320k in the 32-bit VM and 1024k in the 64-bit VM.

In JDK 8, every time the JVM creates a thread, the OS allocates some native memory to hold that thread’s stack, committing more memory to the process until the thread exits. Thread stacks are fully allocated (i.e., committed, not just reserved) when they are created.

This means that if your application spawns a lot of threads, this can consume a significant amount of memory which could otherwise be used by your application or OS (or it can eventually leads to OutOfMemoryError).

You can reduce your stack size by running with the -Xss option. For example:

java -server -Xss256k

or

java -server -XX:ThreadStackSize=256 

Note that if you have installed a 64-bit VM binary for Linux, you can omit -server option.^[5]

Virtual Memory Map

In JDK 8, HotSpot installation comes with a feature named Native Memory Tracking (default: disabled).  To enable it, use:

-XX:NativeMemoryTracking=[off|detail|summary]

After enabling NMT, you can examine the memory footprint taken by either Thread or Thread Stack using:

jcmd <pid> VM.native_memory [summary | detail | baseline | summary.diff | detail.diff | shutdown] [scale= KB | MB | GB]

 For example, on a 64-bit Linux platform, here is the thread stack size before and after setting -Xss256k:

Before

 Virtual memory map:

[0x0000000040049000 - 0x000000004014a000] reserved and committed 1028KB for Thread Stack from
    [0x00002aec741ca5e4] JavaThread::run()+0x24
    [0x00002aec74083268] java_start(Thread*)+0x108

After

Virtual memory map:

[0x0000000040078000 - 0x00000000400b9000] reserved and committed 260KB for Thread Stack from
    [0x00002b02c69156e4] JavaThread::run()+0x24
    [0x00002b02c67ce338] java_start(Thread*)+0x108

Conclusions

The thread stack is used to push stacks frames in nested method calls. If the nesting is so deep that the thread runs out of space, the thread dies with a StackOverflowError.^[8] If your applications use lots of recursive algorithms or if your applications are built on top of a framework utilizing MVC design pattern such as Oracle ADF, you may want to leave StackThreadSize as defaults.

However, thread stacks are quite large, particularly for a 64-bit JVM.  In [9], Scott Oaks has advised:

• As a general rule, many applications can actually run with a 128 KB stack size in a 32-bit JVM, and a 256 KB stack size in a 64-bit JVM.
• In a 64-bit JVM, there is usually no reason to set this value unless the machine is quite strained for physical memory and the smaller stack size will prevent applications from running out of native memory. 
• On the other hand, using a smaller (e.g., 128 KB) stack size on a 32-bit JVM is often a good idea, as it frees up memory in the process size and allows the JVM to utilize a larger heap.

Finally, the total footprint of the JVM has a significant effect on its performance. So, footprint is one aspect of Java performance that should be commonly monitored.

References

1. What Are the Default HotSpot JVM Values? 
2. JDK 8: UseCompressedClassPointers vs. UseCompressedOops
3. The Java Stack 
4. Frequently Asked Questions About the Java HotSpot VM 
5. HotSpot VM Binaries: 32-Bit vs. 64-Bit 
6. How to Debug Native OutOfMemory in JRockit 
7. HotSpot: Monitoring and Tuning Metaspace in JDK 8
8. Using -Xss to adjust Java default thread stack size to save memory and prevent StackOverflowError
9. Java Performance: The Definitive Guide (Strongly recommended)
10. Java thread stack size on 64-bit linux 
11. Learn More About Performance Improvements in JDK 8 
12. HotSpot Virtual Machine Garbage Collection Tuning Guide
13. Getting Started with the G1 Garbage Collector
14. Other JDK 8 articles on Xml and More
15. Tuning that was great in old JRockit versions might not be so good anymore
    
• Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
• Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).
16. HotSpot VM options (JDK 8)

Understanding JVM Thread States

To investigate CPU issues in Java applications, one approach is to diagnose monitor locks and thread activities^[1].

In this article, we will show you:

• How to generate thread dumps on different VMs
• What to know about thread states

JVM Thread States

A thread can be in only one state at a given point in time. These states are virtual machine states which do not reflect any operating system thread states. As an example, here shows different threads in different states on a HotSpot VM:

$ cat thread.tmp | grep "java.lang.Thread.State" | sort | uniq -c
      3    java.lang.Thread.State: BLOCKED (on object monitor)
     18    java.lang.Thread.State: RUNNABLE
      6    java.lang.Thread.State: TIMED_WAITING (on object monitor)
      2    java.lang.Thread.State: TIMED_WAITING (sleeping)
     13    java.lang.Thread.State: WAITING (on object monitor)
      3    java.lang.Thread.State: WAITING (parking)

Below we describe some of the thread states that can be found in a thread dump:

• NEW - this state represents a new thread which is not yet started.
• RUNNABLE - this state represents a thread which is executing in the underlying JVM. Here executing in JVM doesn't mean that the thread is always executing in the OS as well - it may wait for a resource from the Operating system like the processor while being in this state.
• BLOCKED (on object monitor)- this state represents a thread which has been blocked and is waiting for a moniotor to enter/re-enter a synchronized block/method. A thread gets into this state after calling Object.wait method.
• WAITING - this state represnts a thread in the waiting state and this wait is over only when some other thread performs some appropriate action. A thread can get into this state either by calling - Object.wait (without timeout), Thread.join (without timeout), or LockSupport.park methods.
• TIMED_WAITING - this state represents a thread which is required to wait at max for a specified time limit. A thread can get into this state by calling either of these methods: Thread.sleep, Object.wait (with timeout specified), Thread.join (with timeout specified), LockSupport.parkNanos, LockSupport.parkUntil
• TERMINATED - this state reprents a thread which has completed its execution either by returning from the run() method after completing the execution OR by throwing an exception which propagated from the run() method and hence caused the termination of the thread.
• WAITING (parking)- it means a wait state after being parked. A thread can suspend its execution until permit is available (or thread is interrupted, or timeout expired, etc) by calling park(). You can give permit to a thread by calling unpark(). When permit is available, the parked thread consumes it and exits a park() method. Unlike Semaphore's permits, permits of LockSupport are associated with threads (i.e. permit is given to a particular thread) and doesn't accumulate (i.e. there can be only one permit per thread, when thread consumes the permit, it disappears).

Notes:

1. It's hard to investigate any CPU issue just from one thread dump. So, you need to prepare a series of thread dumps for investigation. For an example of analyzing hanging problems, see [9].
2. The above thread state information is for HotSpot.  To understand JRockit's thread dump contents, read [4].

Generating Thread Dumps in HotSpot

$jcmd 15679 Thread.print >thread.tmp

where 15679 is the process ID. For example, it will print the following messages:

15679:
2012-08-24 11:27:27
Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-b21-internal mixed mode):

"Attach Listener" daemon prio=10 tid=0x000000000528e000 nid=0x4653 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Thread-29" daemon prio=10 tid=0x00002aaab9447000 nid=0x3fe3 runnable [0x000000004408b000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:150)
        at java.net.SocketInputStream.read(SocketInputStream.java:121)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        - locked <0x00000000f5ae4738> (a java.io.BufferedInputStream)
        at com.sun.jndi.ldap.Connection.run(Connection.java:849)
        at java.lang.Thread.run(Thread.java:722)

Generating Thread Dumps in JRockit^[4]

$jrcmd 1286 print_threads

where 1286 is the process ID. For example, it will print the following messages:

1286:

===== FULL THREAD DUMP ===============
Thu Aug 23 17:36:30 2012
Oracle JRockit(R) R28.1.3-11-141760-1.6.0_24-20110301-1432-linux-x86_64

"Main Thread" id=1 idx=0x4 tid=1287 prio=5 alive, waiting, native_blocked
    -- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0xe48b4598[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:485)
    at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:981)
    ^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0xe48b4598[fat lock]
    at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:490)
    at weblogic/Server.main(Server.java:71)
    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
    -- end of trace

"(Signal Handler)" id=2 idx=0x8 tid=1288 prio=5 alive, native_blocked, daemon

"(OC Main Thread)" id=3 idx=0xc tid=1289 prio=5 alive, native_waiting, daemon
...

References

1. Understanding Threads and Locks
2. Thread States Diagram
3. Useful tool: jrcmd
4. Using Thread Dumps
5. Understanding a Java thread dump
6. Fun with JStack 
7. Java 2 Platform, Standard Edition 5.0 "Trouobingshooting and Diagnostic Guide"
8. HotSpot VM Performance Tuning Tips
9. Analyze Hanging Programs Using Java Thread Traces (XML and More)
10. Analyzing Thread Dumps in Middleware - Part 1
11. Analyzing Thread Dumps in Middleware - Part 2