JRockit: Could not acquire large pages for 2Mbytes code

When we tried to enable Large Pages for JRockit, we have seen the following message from the WebLogic Server console output:

• WARN codegc Could not acquire large pages for 2Mbytes code (at 0x2aaab0622000).

In this article, we will show you how to investigate and resolve this issue.

How to Test for Large Pages Support?

Similar to [1], here are the VM options for testing large-pages support for JRockit:

$  bin/java -Xms2560m -Xmx2560m -XlargePages -Xgc:genpar -XlargePages:exitOnFailure -version

When we ran the above command, we have seen the following warning:

[WARN ][codegc ] Could not acquire large pages for 2Mbytes code (at 0x2aaab0622000).
[WARN ][codegc ] Falling back to normal page size.
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
Oracle JRockit(R) (build R28.2.0-79-146777-1.6.0_29-20111005-1807-linux-x86_64, compiled mode)

Enabling Large Pages Support in Linux Kernel

We have followed the procedure described in [2,5] to enable huge-pages support in the Linux kernel.  One of the requirements is to mount hugepages directory on the hugetlbfs type filesystem^[6] (note that this step is required for JRockit, but not HotSpot).  For example, we have hugepages directory mounted as follows:

$ mount -l
nodev on /mnt/hugepages type hugetlbfs (rw,noexec,nosuid,nodev,sync,uid=59951)

How to Investigate?

When we tried to explicitly disabled large pages for Java code (but not Java heap) , the following VM options ran fine:

$  bin/java -Xms2560m -Xmx2560m -XlargePages -Xgc:genpar -XlargePages:exitOnFailure -XX:+UseLargePagesForHeap -XX:-UseLargePagesForCode -XX:+FlightRecorder -XX:FlightRecorderOptions=defaultrecording=false  -version
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
Oracle JRockit(R) (build R28.2.0-79-146777-1.6.0_29-20111005-1807-linux-x86_64, compiled mode)


So, the issue is related to Java code reservation in Huge Pages.  After further investigation, we have found the real problem is that: when we mounted hugepages directory, we have chosen the following option:

• noexec
• Do not allow direct execution of any binaries on the mounted filesystem

After we have removed that constraint and rebooted the system, we have finally resolved the issue as shown below:

$  bin/java -Xms2560m -Xmx2560m -XlargePages -Xgc:genpar -XlargePages:exitOnFailure -XX:+UseLargePagesForHeap -XX:+UseLargePagesForCode -XX:+FlightRecorder -XX:FlightRecorderOptions=defaultrecording=false -version
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
Oracle JRockit(R) (build R28.2.0-79-146777-1.6.0_29-20111005-1807-linux-x86_64, compiled mode)

Here are the settings of newly mounted hugepages directory:

$ mount -l
nodev on /mnt/hugepages type hugetlbfs (rw)

Acknowledgement

This issue was resolved based on the feedbacks from Scott Oats.

References

1. How to Test Large Page Support on Your Linux System?
2. Java SE Tuning Tip: Large Pages on Windows and Linux 
3. Oracle® JRockit Command-Line Reference Release R28
4. Memlock limit too small (one of the requirements for large page support)
5. How to acquire large pages for Java heap
6. Linux / Unix Command: mount
7. Oracle® JRockit Performance Tuning Guide Release R28