Thursday, October 23, 2014

WebLogic: How to Create a WLS Domain?

In WebLogic Server, you can create a domain using:[1]
  • WebLogic Configuration Wizard
  • WLST
  • weblogic.Server (a command-line utility)
Your actual choice of a domain creation tool depends on whether you prefer a graphical or command-line interface, and whether you need to automate the domain creation process. Using above-mentioned tools, you can either create a domain or extend an existing domain.

In this article, we will look at two ways to create a WebLogic domain:
  • Creating a new domain based on an existing domain
  • Creating a new domain based on a domain template

Domain Template

A template is simply a jar file that contains the necessary scripts and files WebLogic Server needs to create or extend domains.

There are three basic types of templates in WebLogic Server:[1]
  • Domain Templates
    • Help create an entire domain
      • Defines the full set of resources within a WebLogic domain, including:
        • Infrastructure components
        • Applications
        • Services
        • Security options
        • General environment and operating system options.
  • Extension Templates
    • Add to the functionality of a domain
      • Oracle uses special extension templates for various Fusion Middleware products.
  • Managed Server Templates
    • Defines the subset of resources within a WebLogic domain that are required to create a Managed Server domain directory on a remote machine. 
You can create the first two types of templates with the Domain Template Builder,[2] and you can create all three types with the pack utility.[3]

Creating a New Domain Based on an Existing Domain

Follow these steps to create a new domain using WLST offline commands:
  1. Execute the readDomain command to open an existing WebLogic domain
    • wls:/offline>readDomain('C:/Oracle/Middleware/user_projects/domains/base_domain')
    • wls:/offline/base_domain>
  2. Execute the writeTemplate command to write the current domain configuration to a specific domain template.
    • wls:/offline/base_domain>writeTemplate('C:/Oracle/Middleware/user_templates/MyBaseTemplate.jar')
    • wls:/offline/base_domain>
  3. Execute the closeDomain command to close it
    • wls:/offline/base_domain>closeTemplate()
    • wls:/offline/base_domain>
  4. Execute writeDomain command to create a new domain
    • wls:/offline/base_domain>createDomain('C:/Oracle/Middleware/user_templates/ws_server_12.1.2.0.jar', 'C:/Oracle/Middleware/user_projects/domains/my_domain1','weblogic','welcome1')
    • wls:/offline/base_domain>
You won’t receive any confirmation that the domain has been created, but the absence of any error messages means that the domain was successfully created. Note that the createDomain WLST command works essentially the same as the unpack command.[3]

Creating a New Domain Based on a Domain Template

Probably the easiest way to create a domain is through the Configuration Wizard. The Configuration Wizard uses default templates to create or extend an existing domain.

Follow these steps to create a new domain using Configuration Wizard on Windows:[1,7,8]
  1. Click the Windows shortcut at Start | Programs | Oracle | OracleHome | WebLogic Server 12c | Tools | Configuration Wizard
  2. Select Create A New Domain On the Create Domain screen
  3. Choose one of the following two options on the Templates screen
    • Create Domain Using Product Templates
    • Create Domain Using Custom Template
  4. Specify the login credentials for this new domain on the Administrator Account screen
  5. Configure startup mode and JDK on the Domain Mode And JDK screen
  6. Configure the database schemas on the Configure JDBC Data Sources screen
  7. Specify keystore password on the Configure Keystore Credentials screen
  8. Perform certain advanced configuration tasks on the Advanced Configuration screen
  9. Review your domain configuration settings on the Configuration Summary screen
  10. Check Configuration Progress and Status
You can use Configuration Wizard to create a domain with just an Admin Server or a domain that includes additional Managed Servers or even a cluster. In addition, you can customize a domain by configuring JDBC settings, for example, to point to a different database rather than the default database (Apache Derby database) used by the domain and extension templates.

Avitek Medical Records Sample Domain

Avitek Medical Records Sample Domain Template is one of the WebLogic domain templates that are provided with your WebLogic Server installation.[5] You can use it to extend the basic WebLogic Server domain, which creates the Avitek Medical Records sample domain.

This domain is a WebLogic Server sample application suite that demonstrates all aspects of the J2EE platform. Read [5] to view the details of generated domain output.


  1. Oracle WebLogic Server 12c Administration Handbook
  2. Using the Domain Template Builder to Create Domain Templates
  3. Overview of the Pack and Unpack Commands
  4. WebLogic Server Templates (12c)
  5. Avitek Medical Records Sample Domain Template
  6. The Configuration File in WebLogic Server Domain — config.xml (Xml and More)
  7. Configuring Your Oracle Service Bus Domain
  8. WebLogic Server: Installation to Deployment in 30 Minutes
    • To start the wizard on Windows, type:
      • %WL_HOME%\common\bin\config.cmd
    • To start the new domain, do:
      • %MW_HOME%\user_projects\domains\dev_domain\startWebLogic.cmd
  9. WebLogic Server 12c and WebLogic Server 11g Releases (OTN)
  10. Configuration Wizard Screens (12.1.2)

Wednesday, October 15, 2014

Excel: How to Adjust the Scale of the Axis in a Scatter Chart

Routinely, we have been using TextPad and Excel to analyze Garbage Collection (GC) events gathered in GC log files.

In this article, we will demonstrate how to use these data to create a Scatter Chart and also adjust the scale of the horizontal axis in the chart as shown below:

Before Scale Change

After Scale Change

The Data

After cleaning the data, we have two columns holding data as shown below:
  • GC time stamp
  • Mixed GC pause time
Using shift key, you can select both A1 and B1 cells as shown below:

Then you can select entire columns by pressing:[1]

Create a Scatter Chart from Spreadsheet Data

A scatter chart has two value axes, showing one set of numerical data along the x-axis and another along the y-axis. It combines these values into single data points and displays them in uneven intervals, or clusters.

To create a scatter chart, you do:[2]
  1. Arrange your data so that the x-values are in the first column of your worksheet, and the y-values are located in adjacent columns.
  2. Select the range of x- and y-values that you want to plot in the chart.
  3. Click Chart on the Insert menu to start the Chart Wizard.
  4. In the Chart type box, select Scatter .
  5. Under Chart sub-type, click the chart sub-type you want to use.

This will create a scatter chart as shown in diagram 1.  But, you can see the scale of X-axis is not appropriate.

Change the Scale of Horizontal Axis

By default, Microsoft Office Excel determines the minimum and maximum scale values of the axis when you create a chart. However, you can customize the scale to better meet your needs.  To customize the scale of horizontal axis, you do:
  1. Change the label/title of the series to "Mixed GC"[4]
  2. In the chart, click the horizontal (value) axis that you want to change.
  3. This displays the Chart Tools, adding the Design, Layout, and Format tabs.
  4. On the Format tab, in the Current Selection group, click the arrow next to the Chart Elements box, and then click Horizontal (Value) Axis.
  5. In the Current Selection group, click Format Selection.
  6. In the Format dialog box, change maximum value of X-axis from 40000.0 to 30000.0.
Voila, you have created a scatter chart of the Mixed GC diagram with a good scale at horizontal axis (see diagram 2 above).

Sunday, October 12, 2014

JDK 8: Is Tuning MaxNewSize in G1 GC a Good Idea?

For a throughput GC such as Parallel GC, it often helps performance by setting MaxNewSize (or -Xmn).  For G1 GC,[1] is it the same? The short answer is yes/no and it depends (confession: I said "no" because I haven't done enough tests to exclude it).

In this article, we will demonstrate one case that setting MaxNewSize for G1 actually hurts the application's performance.  BTW, after benchmarking G1 in JDK 7, another author has claimed that:[14]
  • There is little to gain and much to lose from setting the new generation size explicitly.

Live Data Size

To tune the heap size of a Java Application, it's important to find out what its live data size is.  To learn how to estimate a Java application's live data size, read [2].  For the benchmark we are using, its live data size is around 1400 MB. With this benchmark, we are interested in knowing how G1 performs with MaxNewSize set to 400MB and Java heap set to 2GB.

Regularly we use the following settings for the G1 tuning:
  • -Xms2g -Xmx2g -XX:+UseG1GC
plus some options for mixed GC (read [3]). So, for this experiment, we have added MaxNewSize setting like this:
  • -Xms2g -Xmx2g -XX:+UseG1GC -Xmn400m
This has turned out hurting the performance a lot. Note that -Xmn400m is the optimal setting of our benchmark if Parallel GC is used.

Concurrent Cycle in G1 GC

G1 has four main operations:[5]
  • A young collection
  • A background, concurrent cycle
  • A mixed collection
  • If necessary, a full GC

Without much ado, read [5-9] for the needed background of G1 GC.

G1 is a concurrent collector.  Its concurrent cycle has multiple phases—some of which stop all application threads (denoted by STW) and some of which do not:
  • Initial-mark (STW)
    • Denoted by: 
      • [GC pause (G1 Evacuation Pause) (young) (initial-mark)
      • [GC pause (Metadata GC Threshold) (young) (initial-mark)
    • The G1 GC marks the roots during this phase. This phase is piggybacked on a normal (STW) young garbage collection.
  • Root Region Scan
    • Denoted by: 
      • [GC concurrent-root-region-scan-start]
      • [GC concurrent-root-region-scan-end
    • The G1 GC scans survivor regions of the initial mark for references to the old generation and marks the referenced objects. 
    • This phase runs concurrently with the application (not STW) and must complete before the next STW young garbage collection can start.
  • Concurrent marking
    • Denoted by:
      • [GC concurrent-mark-start]
      • [GC concurrent-mark-end
    • The G1 GC finds reachable (live) objects across the entire heap. 
    • This phase happens concurrently with the application, and can be interrupted by STW young garbage collections.
  • Remark (STW)
    • Denoted by:
      • [GC remark
    • This phase is STW collection and helps the completion of the marking cycle. 
      • Finds objects that were missed by the concurrent mark phase due to updates by Java application threads to objects after the concurrent collector had finished tracing that object.
    • G1 GC drains SATB buffers, traces unvisited live objects, and performs reference processing.
  • Cleanup (STW)
    • Denoted by:
      • [GC cleanup 1936M->1931M(2048M)
    • Prepare for next concurrent collection by clearing data structures.
      • In this final phase, the G1 GC performs the STW operations of accounting and RSet scrubbing. 
      • During accounting, the G1 GC identifies completely free regions and mixed garbage collection candidates. The cleanup phase is partly concurrent when it resets and returns the empty regions to the free list.
  • Concurrent cleanup:
    • Denoted by:
      • [GC concurrent-cleanup-start]
      • [GC concurrent-cleanup-end
    • In this phase, G1 reclaims regions which were found empty during marking.
      • Adds empty regions to the free list—i.e., thread local free lists are merged into global free list

G1 GC uses the Snapshot-At-The-Beginning (SATB) algorithm, which takes a snapshot of the set of live objects in the heap at the start of a marking cycle.  During the concurrent cycle, young garbage collections are allowed, which are triggered when eden fills up (note that initial-mark is implemented using a young collection cycle) .

The set of live objects after marking cycle is composed of the live objects in the snapshot, and the objects allocated since the start of the marking cycle. The G1 GC marking algorithm uses a pre-write barrier to record and mark objects that are part of the logical snapshot.

After the concurrent cycle, we expect to see:[5]
  • The eden regions before the marking cycle have been completely freed and new eden regions have been  allocated
  • Old regions could be more occupied because the promotion of live objects from young regions
  • Some old regions are identified to be mostly garbage and become candidates in later mixed or old collection cycles


The reason of setting -Xmn400m has caused the regression of G1 is:
The option has forced G1 to use a young generation space of up to 400 MB and leaves G1 just 200 MB (i.e. 2048MB - 400MB - 1400MB) breathing room for shuffling live objects around.    
The small free space has negative impact on G1's concurrent cycles, which has caused most marking cycles not being completed in time.  
Before setting -Xmn400m on the command line, we have found only one instance of aborted marking cycle, which is denoted by:
  • [GC concurrent-mark-abort]
After setting it, we have seen marking cycle was aborted 105 times out of 150 instances.  The lesson we have learnt here is that for a tight heap like our benchmark, it's not a good idea to set -Xmn400m for G1.  Without setting MaxNewSize, G1's ergonomics will dynamically adjust young generation space and it turns out that G1 can do a better job in this case.

Differences between Parallel GC and G1 GC

Note that -Xmn400m is the optimal setting for our benchmark if Parallel GC is used.  When we set MaxNewSize to be 400MB for G1, our benchmark regressed.  So, what's the difference between G1 GC and Parallel GC?

Initially, G1 GC was designed to replace CMS [10] for its relatively lower and more predictable pause times.  This is different from the design of Parallel GC which aims for higher throughput.  To help gain higher throughput, Parallel GC tries to:
  • adjust young generation as large as possible and let it run into a full GC
    • In our 4-hour experiments, we usually see around 50 full GC's in Parallel GC
      • However, for G1's performance tuning, we want to avoid full GC's (see [11] for how)
    • Which means more frequent full GC is expected
      • Note that Parallel GC's Full GC pause time is shorter than G1's in the current JDK 8 releases
To conclude, if you upgrade JDK or switch garbage collector,[7] always re-evaluate original optimal VM settings.


Some writings here are based on the feedback from Thomas Schatzl. However, the author would assume the full responsibility for the content himself.


  1. Garbage First Garbage Collector Tuning
  2. JRockit: How to Estimate the Size of Live Data Set
  3. g1gc logs - basic - how to print and how to understand
  4. g1gc logs - Ergonomics -how to print and how to understand
  5. Java Performance: The Definitive Guide (Strongly recommended)
  6. Garbage First Garbage Collector Tuning - Oracle
  7. Our Collectors by Jon Masamitsu
  8. Understanding Garbage Collection
  9. HotSpot VM Performance Tuning Tips
  10. Understanding CMS GC Logs
  11. G1 GC: Tuning Mixed Garbage Collections in JDK 8
  12. G1 GC Glossary of Terms
  13. Learn More About Performance Improvements in JDK 8 
  14. Benchmarking G1 and other Java 7 Garbage Collectors
  15. HotSpot Virtual Machine Garbage Collection Tuning Guide
  16. Getting Started with the G1 Garbage Collector
  17. Garbage-First Garbage Collector (JDK 8 HotSpot Virtual Machine Garbage Collection Tuning Guide)
  18. Other JDK 8 articles on Xml and More
  19. Tuning that was great in old JRockit versions might not be so good anymore
    • Trying to bring over each and every tuning option from a JR configuration to an HS one is probably a bad idea.
    • Even when moving between major versions of the same JVM, we usually recommend going back to the default (just pick a collector and heap size) and then redoing any tuning work from scratch (if even necessary).

Thursday, October 9, 2014

New JDK 8 support in WebLogic Server

Oracle has just announced that Oracle WebLogic Server 12.1.3 has been certified on Java SE 8. It is supported on:

  • Windows
  • Linux
  • Solaris 64-bit 

platforms with HotSpot JDK8 Update 20 or later.

If you want to learn more about what is supported and what are the limitations, check:

Friday, October 3, 2014

OATS: Tie All Processes Together — from OpenScript to Scenario

We have been blogging different topics on OATS since 2013.  In this article, we'll tie all processes together.

In [1], we have demonstrated how to create an open script for load testing using CRM FUSE (i.e. a Fusion application).  Assume you have done that, the next step will be testing by clicking on the  Playback button:

If the test succeeds, you should see Passed status in the Results.  Since we have hard coded user name (i.e., SALESREPUSER00001) in the Post Data step, the next step will be setting up databanks in OpenScript and then configuring scripts to use databanks.  See [2] for details.

But, before we do that,  I usually clean up steps a bit by removing steps that loaded cookies or images (i.e., png files).  In my experience, they were not useful.  Since we will migrate OpenScript file from Windows to Linux, the cookie recorded might be useless.  Also, we would like the whole zip file self-contained.  By removing image files, we don't need to worry about the dependency of script to the images (i.e., where to load them from).  Anyway, before you remove anything from the project.  Save the project and make a backup first.

After deleting steps involving either cookie or images, you need to playback steps to make sure everything still works OK.  You then proceed to iterate the test using multiple unique users.  To create databanks needed for the test, we have copied it from a previous project.  Noted there is a new folder named databanks in our new project folder (i.e., FUSE_Saleopty_oct01_wrk).

Inside the databanks folder, there are two files:
  • UserName.csv
  • UserName.csv.index3

You may delete the index file (i.e., UserName.csv.index3) because it will be re-generated by OpenScript later.  Inside UserName.csv, you will find the following entries:


Then you follow [2] to substitute a variable for the hard coded user name (i.e., SALESREPUSER00001).  Make sure you have selected the correct csv file (i.e., the one under your new project folder).   Note that we have set up all users to share the same password in WebLogic Server. 

Now, we can test the script with multiple virtual users by clicking on "Iterate (Alt+I)" button:

Let's iterate the test for no more than 3 times:

After the testing, you then export a self-contained zip file (note that we have excluded some unnecessary data from the export):

Read [3] for more details on how to export and import scripts from Windows to Linux.  To import OpenScript into Oracle Load Testing (OLT which is run on Linux in our case), you start up OLT and select Tools > Import...

Note that when you migrate OpenScript zip file from Windows to Linux, copy it somewhere outside the $installDir/OFT folder. On the Import File dialog, open it and specify Default Repository as  your destination which is the OFT folder.

After import the OpenScript, you can find it from the Default folder (highlighted on the diagram).  If not, exit OLT and restart it.  Clicking on the script named FUSE_Saleopty_oct01_wrk, this will allow you start building a new scenario.  However, with the default setup, you can submit the scenario and start a test immediately:

Let the test run for a while and then click the same button to stop it.  Note that it will take a while before the test fully stops.  If you don't see any Failed transaction, it means your script is OK.  Then the next step is to configure parameters by selecting "Build Scenarios" tab.  This will bring you back to the window before you started the test.  For advanced editing, you can click on the "Configure all parameters" icon.  This allows you to edit scenario details.

For our tests, we have set the following:
  • # VUs: 200
  • VU Pacing (Think Time): Recorded/Random
    • Lower: 15 (percent)
    • Upper: 185 (percent)
which mean OLT will generate 200 virtual users using recorded delays, but with some variations (i.e., randomly ranged from 15% to 185% of the original delay time).  The next step is to "Set up Autopilot".  After clicking on the tab, this will allow you to control the timing and other advanced features:

In our tests, we only need to set up stop time (i.e., after 3 hours and 20 minutes).  Note that our benchmark utilizes this new scenario in a refworkload framework, which is composed of a few Linux shell scripts that drive the whole process.  For this reason, we didn't use other advanced features such as ServerStats or Scenario Profiles as shown above.

Finally, to save your new scenario, select Scenario > Save as.  Be warned that this writing is based on an old installation (i.e., OATS Build 376) and might be a bit different from newer versions.[4]


  1. How to Create Load Testing Scripts Using OpenScript (Xml and More)
  2. How to Configure Scripts to use Databanks in OATS (Xml and More)
  3. OATS: How to Export and Import Scripts from Windows to Linux
  4. Oracle Application Testing Suite
  5. Auto-Correlating Session IDs in Oracle Application Test Suite (Xml and More)