JMeter—Use Extractors (Post-Processor Elements) for Correlation

What Is Correlation

Correlation is the process of capturing and storing the dynamic response (e.g., "Session ID" in the above diagram) from the server and passing it on to subsequent requests. A response is considered dynamic when it can return different data for each iterating request, occasionally affecting successive requests. Correlation is a critical process during performance load test scripting, because if it isn’t handled correctly, your script will become useless.

Correlation is 2-step process:

1. Parse and extract the dynamic value from the response of a step using a Post Processor element such as:
• Regular Expression Extractor
• CSS/JQuery Extractor
2. Refer the extracted value in the request of a subsequent step
• http://.../.../...?sessionID=${sessionId}#....

How to Use Regular Expression Extractor

Watching above video, you can learn how to create Regular Expression Extractors in JMeter in the following steps:^[2]

1. Create a Test Plan where you want to do dynamic referencing in JMeter
2. Add Regular Expression Extractor in the Step from where response value(s) needs to be extracted
• You can use RegExr—an online tool—to learn, build, and test Regular Expressions
3. Refer the extracted value (referred by Reference Name) in subsequent step(s)
4. Run and validate it
• You can use RegExp Tester in View Results Tree, to test regular expressions directly on response data

How to Use CSS/JQuery Extractor

Watching above video, you can learn how to create CSS/JQuery Extractors in JMeter in similar steps:^[8]

1. Create a Test Plan where you want to do dynamic referencing in JMeter
2. Add CSS/JQuery Extractor^[9]in the Step from where response value(s) needs to be extracted
• You can find a detailed explanation of CSS syntax here. jQuery's selector engine uses most of the same syntax as CSS with some exceptions. For selecting an arbitrary locator, you can use field Match No. with the ‘0’ value, which returns a random value from all found results.
• It is also worth mentioning there is a list of very convenient browser plugins to test CSS locators right into your browser. For Firefox, you can use the ‘Firebug’ plugin, while for Chrome ‘XPath Helper’ is the most convenient tool.
3. Refer the extracted value (referred by Reference Name) in subsequent step(s)
4. Run and validate it

How to Use JSON Extractor

Read the companion articles on this subject:

• JMeter: How to Verify JSON Response?
• JMeter―Select on Multiple Criteria with JSONPath

References

1. Advanced Load Testing Scenarios with JMeter: Part 1 - Correlations
2. JMeter Beginner Tutorial 19 - Correlation (with Regular Expression Extractor)
3. Using RegEx (Regular Expression Extractor) with JMeter
4. RegExr—an online tool (good)
5. JMeter Listeners - Part 1: Listeners with Basic Displays
6. Understand and Analyze Summary Report in Jmeter
7. How to Automate Auth Token using JMETER
8. How to Use the CSS/JQuery Extractor in JMeter (BlazeMeter)
9. How to Use the CSS/jQuery Extractor in JMeter  (DZone)
10. JMeter: How to Turn Off Captive Portal from the Recording Using Firefox (Xml and More)
11. JMeter―Select on Multiple Criteria with JSONPath  (Xml and More)
12. JMeter: How to Verify JSON Response?  (Xml and More)

HotSpot: Using jstat to Explore the Performance Data Memory

HotSpot provides jvmstat instrumentation for performance testing and problem isolation purposes.  And it's enabled by default (see -XX:+UsePerfData).

If you run Java application benchmarks, it's also useful to save PerfData memory to hsperfdata_ file on exit by setting:

• -XX:+PerfDataSaveToFile

A file named  hsperfdata_<vmid> will be saved in the WebLogic domain's top-level folder.

How to Read hsperfdata File?

To display statistics collected in PerfData memory, you can use:

• jstat^[3]
• Experimental JVM Statistics Monitoring Tool - It can attach to an instrumented HotSpot Java virtual machine and collects and logs performance statistics as specified by the command line options. (formerly jvmstat)

There are two ways of showing statistics collected in PerfData memory:

• Online
• You can attach to an instrumented HotSpot JVM and collect and log performance statistics at runtime.
• Offline
• You can set -XX:+PerfDataSaveToFile flag and read the contents of the hsperfdata_ file on the exit of JVM.

In the following, we have shown an offline example of reading the hsperfdata_ file (i.e. a binary file; you need to use jstat^[3] to display its content):
$ /scratch/perfgrp/JVMs/jdk-hs/bin/jstat -class file:///<Path to Domain>/MyDomain/hsperfdata_9872

Loaded    Bytes  Unloaded   Bytes       Time
30600   64816.3         2     3.2      19.74

You can check all available command options supported by jstat using:

$jdk-hs/bin/jstat -options
-class
-compiler
-gc
-gccapacity
-gccause
-gcmetacapacity
-gcnew
-gcnewcapacity
-gcold
-gcoldcapacity
-gcutil
-printcompilation

HotSpot Just-In-Time Compiler Statistics

One of the command option supported by jstat is "-compiler", which can provide high-level JIT compiler statistics.

Column	Description
Compiled	Number of compilation tasks performed.
Failed	Number of compilation tasks that failed.
Invalid	Number of compilation tasks that were invalidated.
Time	Time spent performing compilation tasks.
FailedType	Compile type of the last failed compilation.
FailedMethod	Class name and method for the last failed compilation.

In the following, we have shown the compiler statistics of three managed servers in one WLS Domain using two different JVM builds:

$/scratch/perfgrp/JVMs/jdk-hs/bin/jstat -compiler file:///<Path to Domain>/MyDomain/hsperfdata_9872


JVM1

Compiled Failed Invalid   Time   FailedType FailedMethod
   33210     13       0   232.97          1 oracle/ias/cache/Bucket objInvalidate
   74054     20       0   973.03          1 oracle/security/o5logon/b b
   74600     18       0  1094.21          1 oracle/security/o5logon/b b

JVM2

Compiled Failed Invalid   Time   FailedType FailedMethod
   33287     10       0   246.26          1 oracle/ias/cache/Bucket objInvalidate
   68237     18       0  1022.46          1 oracle/security/o5logon/b b
   67346     18       0   943.79          1 oracle/security/o5logon/b b

Given the above statistics, we could take next action on analyzing why JVM2 generating less compiled methonds than JVM1 did. At least this is one of the use case for using PerfData with its associated tool—jstat.

PerfData-Related JVM Options

Name	Description	Default	Type
UsePerfData	Flag to disable jvmstat instrumentation for performance testing and problem isolation purposes.	true	bool
PerfDataSaveToFile	Save PerfData memory to hsperfdata_ file on exit	false	bool
PerfDataSamplingInterval	Data sampling interval in milliseconds	50 /*ms*/	intx
PerfDisableSharedMem	Store performance data in standard memory	false	bool
PerfDataMemorySize	Size of performance data memory region. Will be rounded up to a multiple of the native os page size.	32*K	intx

Note that the default size of PerfData memory is 32K. Therefore the file (i.e., hsperfdata_ file) dumped on exit is also 32K in size.

References

1. New Home of Jvmstat Technology
2. The most complete list of -XX options for Java JVM
3. jstat - Java Virtual Machine Statistics Monitoring Tool

On Stack Replacement in HotSpot JVM

In JVM, JIT Compiler is used to convert bytecode at runtime prior to executing it natively.  Since the JIT does not have time to compile every single method in an Java application, all code starts out initially running in the interpreter, and once it becomes hot enough it gets scheduled for compilation.

The HotSpot VM can perform special compiles called On Stack Replacement compiles, or OSRs.  These are used when Java code contains a long-running loop (see the example here) that started executing in the interpreter.

As described in [1], it is important to know about OSR if you want to benchmark Java programs and have put everything in a loop to be run in a method such as main.  You may run into some pitfalls caused by OSR on your benchmark results.  Without further ado, I'll refer you to read [1] for more details.

On Stack Replacement (OSR)^[2]

Normally the way Java code ends up in compiled code is that when invoking a method the interpreter detects that there's compiled code for it, and it dispatches to that instead of staying in the interpreter.  This does not help long-running loops that started in the interpreter since they are not being invoked again.

When a long-running loop is detected at runtime, HotSpot VM requests a compile that starts its execution at the first bytecode of loop instead of starting at the first bytecode in the method.  The resulting generated code takes an interpreter frame as its input and uses that state to begin its execution.  In this way, long-running loops are able to take advantage of compiled code.  The act of the generated code taking an interpreter frame as input to be execution is called On Stack Replace.

How to Detect If OSR Happened?

The diagnostic options -XX:+LogCompilation can emit a structured XML log of compilation related activity during a run of the virtual machine.  Because this is a diagnostic option, to enable it, you need to specify  -XX:+UnlockDiagnosticVMOptions first.  For example, to generate compilation log file, you can specify:

• -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation -XX:LogFile=/<path to log>/logs/<name of log>.log 

From the log file, you can search "osr" to see if any loop in your application has been compiled:

<task_queued compile_id='1' compile_kind='osr' 

  method='java/util/jar/JarFile hasClassPathAttribute ()Z' bytes='275' 

  count='223' backedge_count='57584' iicount='223' osr_bci='157' stamp='0.233' 

  comment='tiered' hot_count='57584'/>
<nmethod compile_id='1' compile_kind='osr' compiler='C2' level='4' 

  entry='0x00002aaaab4d4d40' size='1576' address='0x00002aaaab4d4bd0' 

  relocation_offset='288' insts_offset='368' stub_offset='1072' 

  scopes_data_offset='1120' scopes_pcs_offset='1352' dependencies_offset='1512' 

  nul_chk_table_offset='1520' oops_offset='1096'

  method='java/util/jar/JarFile hasClassPathAttribute ()Z' bytes='275' 

  count='241' backedge_count='62159' iicount='241' stamp='0.237'/>

Every compile is assigned a compile id by the system at the point is enqueued and that's recorded as the compile_id attribute. The above example shows an on stack replacement (OSR) compile where the code is going to be used to replace an already existing activation. It is tagged by the compile_kind attribute as:

•  compile_kind='osr'. 

The 'method' attribute is a string version of the VM name of the method with spaces separating the class, method name signature.

The 'bytes' attribute is number of bytecodes in the method.

'count' is the invocation count as recorded by the method invocation counters. Note that these counters are mainly used for triggering compiles and are not guaranteed to be an accurate reflection of the number of times a method has actually executed. Multiple threads may be updated these counters and sometimes the VM will reset a counter to a lower value to delay retriggering of compiles.

'iicount' is the interpreter invocation count. This is a separate copy of the invocation count which is maintained by the profiling support. Again it's not guaranteed to be accurate since multiple threads may update it but it's never reset so it's reasonably accurate.

The 'backedge_count' attribute is used to detect methods that contain loops and to cause them to get compiled earlier than they would with just an invocation counter. Whenever this counter is incremented by the interpreter it checks it against a threshold, and if it crosses this threshold the interpreter requests a compile of that loop.

'stamp' gives a timestamp for the start time.  Many elements contain time stamps as the end which can be used to order them relative to events in other threads and the measure elapsed time. The time stamp is in seconds since the start of the VM and the start time of the VM is recorded in the hotspot_log element in the time_ms attribute.

References

1. Robust Java benchmarking, Part 1: Issues 
2. Java Performance by Charlie Hunt and Binu John
3. LogCompilation Overview