Monday, March 26, 2012

Controlling Thread Pool Size in WebLogic Server

One of the critical areas that relate to the tuning of WebLogic Server is thread management[1].  In previous versions of WebLogic Server, processing was performed in multiple execute queues. Different classes of work were executed in different queues, based on priority and ordering requirements, and to avoid deadlocks.  However, in WLS 9.0 and above, it uses a single thread pool, in which all types of work are executed.[10] WebLogic Server prioritizes work based on rules you define, and run-time metrics, including the actual time it takes to execute a request and the rate at which requests are entering and leaving the pool.

Self-tuning Thread Pool[7]


WebLogic uses work managers with a variable and self-tuning number of worker threads. By default, the self-tuning thread pool size limit is 400. This limit includes all running and idle threads, but does not include any standby threads. The size of thread pool grows and shrinks automatically to improve throughput.  Measurements are taken every 2 seconds and the decision to increase or decrease the thread count is based on the current throughput measurement versus past values.

Thread Management[4]


If your server has four physical processors, theoretically you only need a thread pool with four threads. When you have more threads than there are CPU resources, the throughput may suffer. However, if your threads often make database connections or call some other long-running tasks where they need to wait, you do want to have more threads around so that the ones that aren't waiting can do some work.

In a 3-tiered architecture, you can also have a situation like this: the clients make requests coming into the application server faster than the database server can handle.  Then the clients keep adding requests on the application server until all its threads are busy, all of which just adds load to the database.  The more load you add to the application server that is overloaded, the worse you make the situation.  In other cases that clients cannot keep all threads on the application server busy and leave some of them idle, you may still lose throughput because the cache will be less efficient when a new thread takes a new request versus when a just-used thread takes a new request.

At any rate, tuning the size of thread pool is challenging and time consuming.  Internally Weblogic Server has many work managers configured for different types of work. If WLS runs out of threads in the self-tuning pool (because of system property -Dweblogic.threadpool.MaxPoolSize) due to being undersized, then important work that WLS might need to do could be starved.  While limiting the self-tuning would limit the default WorkManager and internally it also limits all other internal WorkManagers which WLS uses.  So, leaving that task to WebLogic Server seems to be a wise choice. 

However, there are some cases that we do need to set the size of thread pool manually.  For example, to make performance comparison between two different test cases, you may want to eliminate the thread-pool-size variance from the performance results.  In this article, we will show you how to set up minimum and maximum thread pool sizes and how to examine the results of the settings.

Controlling the Size of Thread Pool


There are different ways of changing the size of thread pool.  One way of doing it is by setting them from the command line:
  • -Dweblogic.threadpool.MinPoolSize=5 -Dweblogic.threadpool.MaxPoolSize=5
By setting both MinPoolSize and MaxPoolSize to be the same value, we have forced WLS to use exactly five worker threads.  In our case, our two test cases will be compared with the same number of worker threads and prevent the self-tuning effects from contaminating our performance results.  In [8], it also tells us how to make similar changes via config.xml.

Threads Page on WLS Console


For a WebLogic Server administrator, the WLS console is indispensable for monitoring running server instances, including the various subsystems such as security, JTA, and JDBC. The Threads page on the WLS console provides information on the thread activity for the current server.  In Figure 1, it shows that there are five Active Execute Threads based on our configuration.  If we didn't configure the thread pool size, you could see number of Active Execute Threads changing dynamically due to WLS' self-tuning activities.


.

Default Execute Queue from Thread Dump


Besides monitoring number of worker threads from the WLS console, you can also examine them from the thread dump as generated from JStack[3].

Unless you've customized the execute queue (or thread pool) that your application gets deployed to, you can look for "Default" execute queue.  In the dump file, you'll look for the threads marked as 'weblogic.kernel.Default' to see what's running.  As work enters an instance of WLS, it is placed in the default execute queue.  This work is then assigned to a worker thread that does the work on it.

$ grep weblogic.kernel.Default threadDump.fod1
"[STANDBY] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00000000202d9800 nid=0x404a in Object.wait() [0x0000000040801000]
"[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x0000000021813800 nid=0x3d13 in Object.wait() [0x000000004c52d000]
"[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaabc0c6800 nid=0x3811 runnable [0x000000004a107000]
"[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaabc0c5000 nid=0x3810 in Object.wait() [0x000000004a008000]
"[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaabc0c1800 nid=0x380f in Object.wait() [0x0000000049f06000]
"[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaabc0db800 nid=0x380e in Object.wait() [0x000000004194e000]
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaabc0bc800 nid=0x380d in Object.wait() [0x000000004184d000]

As shown above, you can find there are seven instances of 'weblogic.kernel.Default (self-tuning)'.  Five of them are active and two of them are in standby.[11]  These five active instances match what we've found as "Active Execute Thread" on the WLS console.

References

  1. Oracle WebLogic Server 11g Administration Handbook by Sam Alapati
  2. Using Work Managers to Optimize Scheduled Work
  3. Fun with JStack by Scott Oaks
  4. Rewritten from personal's email exchanges with Scott Oaks
  5. Monitoring WebLogic Server Thread Pool at Runtime
  6. Understanding JVM Thread States
  7. Self-Tuning Thread Pool
  8. Tuning Default WorkManager - Advantages and Disadvantages
  9. Fusion Middleware Performance and Tuning for Oracle WebLogic Server
  10. Understanding the Differences Between Work Managers and Execute Queues
  11. STANDBY thread (WLS)
    • ACTIVE threads can go to STANDBY when it is deemed that you don’t need that many active threads.
    • But, a STANDBY thread can still be used (without transitioning to ACTIVE) in order to satisfy a min threads constraint.
  12. Top Tuning Recommendations for WebLogic Server (12.2.1.3.0)
  13. Analyzing Thread Dumps in Middleware - Part 2

24 comments:

Manish Kumar Gupta said...

Nice Article, gives high level view of Pool size, thanks for that.

Blogger said...

Did you know that that you can make cash by locking selected sections of your blog or site?
Simply join AdWorkMedia and embed their Content Locking widget.

Nagendra Reddy said...

Actually I need to perform this task in my weblogic training in hyderabad thanks to your post I learn methods how to do this.

Priya Kannan said...

Great site for these post and i am seeing the most of contents have useful for my Carrier.Thanks to such a useful information.Any information are commands like to share him.
Weblogic Training in Chennai

Suresh said...
This comment has been removed by the author.
Anonymous said...

Great site for these post and i am seeing the most of contents have useful for my Carrier. Thanks for the reminder.Thanks for posting useful information.I like it and help me to development very well. gives high level view of Pool size...


WEBLOGIC Training

Unknown said...

Great post! Thanks for sharing with us, Its really gives lot of useful information. Regards,

Angularjs Training in Chennai

Sadhana Rathore said...

Useful content, I have bookmarked this page for my future reference.
R Training in Chennai
R Programming Training in Chennai
R Training in Velachery
RPA Training in Chennai
AWS Training in Chennai
Angularjs Training in Chennai

Anbarasan14 said...

Nice blog. Can't be written much better. You’re doing a great job. Keep continuing.


Spoken English Classes in Coimbatore
Best Spoken English Institute in Coimbatore
Spoken English Training in Coimbatore
Spoken English in Coimbatore
Best Spoken English Courses in Coimbatore
Spoken English Coaching Classes in Coimbatore
Spoken English Classes near me

kaushik said...

Thanks for sharing this valuable information.Its more useful to us.its very interesting to know the blog with clear vision.
Web Designing Course in Bangalore
Web Development Courses in Bangalore
best php training institute in bangalore
php institute in bangalore
best php training in bangalore

Rithi Rawat said...

I think things like this are really interesting. I absolutely love to find unique places like this. It really looks super creepy though!! machine learning training in chennai

artificial intelligence and machine learning course in chennai

machine learning classroom training in chennai

jefrin said...

Great post thanks for sharing
R programming training in chennai

cynthiawilliams said...

I feel happy to see your webpage and looking forward for more updates.
Machine Learning course in Chennai
Machine Learning Training in Chennai
Data Science Course in Chennai
Data Science Classes in Chennai
Data Science Training in Velachery
Data Science Course in Anna Nagar

MindtechAffiliates said...

Thank you very much for this one. And i hope this will be useful for many people.. and i am waiting for your next post keep on updating these kinds of knowledgeable things...Really it was an awesome article...very interesting to read..please sharing like this information.

Thanks
Cpa offers

eddy said...

Very Good Information
PHP Tutorial
What is PHP
How to Install PHP
PHP Examples
PHP Echo
PHP Print
PHP Variables
PHP Constants

svrtechnologies said...


I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly.I want to share about weblogic tutorials .

svrtechnologies said...


I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly. I want to share about weblogic server .

Ramesh Sampangi said...

I feel happy to read this blog. Nice blog and informative. Keep sharing more.
AI Course in Hyderabad

IK Softech said...



Wow. That is so elegant and logical and clearly explained. Brilliantly goes through what could be a complex process and makes it obvious.I want to refer about the best
sap abap training in bangalore . They Provide 100% Placement Program .

Peter Johnson said...

Really thanks for sharing such an useful & nice info..

Agro Fertilizer Company in India

Nathan said...

Very informative Blog! There is so much information here that can help thank you for sharing.
Data Analytics Training in Bangalore

traininginstitute said...

Amazing knowledge and I like to share this kind of information with my friends and hope they like it they why I do
data science course

Kanika said...

Customer Loyalty Rewards Programs Software for small businesses is designed to help small and medium-sized enterprises (SMEs) implement and manage loyalty programs efficiently. These programs are vital for retaining customers, increasing repeat purchases, and fostering brand loyalty.
Best Customer Loyalty Rewards Programs Software for Small Businesses

VISWA Technologies said...

Looking forward to reading more. Great article. Really looking forward to reading more. Keep writing.
ASP .Net MVC Training
Oracle CPQ Training