Book Review: Getting Started with Oracle Event Processing 11g

Event processing is a method of tracking and analyzing (processing) streams of information (data) about things that happen (events),^[1] and deriving a conclusion from them. Complex event processing, or CEP, is event processing that combines data from multiple sources^[2] to infer events or patterns that suggest more complicated circumstances. The goal of complex event processing is to identify meaningful events (such as opportunities or threats)^[3] and respond to them as quickly as possible.

Using airport as an example, events happening in the airport that may require tracking include missing luggages, passenger movements, etc.  An architecture combining Event Driven Architecture (EDA) and Service Oriented Architecture (SOA) can be deployed for such purpose.  A system utilizing Oracle products is illustrated in the following diagram:

In this book, it has covered the following topics and more:

• Product history of Oracle CEP
• Event Driven Architecture
• Requirements of Complex Event Processing
• Creating Event Driven Architecture (EDA) applications using Oracle CEP

History of Oracle Event Processing Solution 11gR1

A product initially named Elvis (or WebLogic Event Server) was developed by the Time and Event Driven (TED) team in early 2007 at BEA Systems Inc. After several "man years" of effort, this technology was first released on July 16, 2007 from BEA Systems.

In parallel, a Complex Event Processing "engine" was researched and developed separately within Oracle.   Starting from 2008, WebLogic Event Server and this CEP "engine" were integrated.  Finally, on July 1, 2009, these combined technologies were released as the Oracle Event Processing Solution 11gR1.  This product is claimed to be one of the most complete and powerful in-memory extensible event analytics technologies available in the market.

Limitations of Traditional Systems for Event Processing

After the emerging of social media, data are continuously created in vast quantities.  This proliferation of data (or Big Data grow) has brought challenges to traditional systems for tracking and analyzing streams of information (data) about things that happen (events).

First thought on supporting events is using traditional systems.  However, they have their limitations:

• Traditional computing systems are mostly based on database
• Information must be inserted and statically placed. Only after that, can users analyze and respond.  
• Frequent disk reads and writes will quickly become system's performance bottlenecks.  
• Traditional JEE Application Servers adopt the send/reply implementation paradigms
• This has introduced delays that won't meet the real-time requirements needed for event processing.

So, a new Event Driven Architecture (EDA) was introduced, which processes event streams in memory using a defined window of time or other filtering options.

Event-Driven (Architecture) Based Systems

Event-driven architecture (EDA)^[5] is a software architecture pattern promoting the production, detection, consumption of, and reaction to events.  Before systems can process any event, they need to normalize the incoming data stream by converting them into events and event types.  These representations are then processed downstream or upstream in the Event Processing Network (EPN).

Before proceeding further, let's introduce you the following definitions:

• Event
• In the context of this book, an event is an object that has a change in its state immediately, or over a period of time.
• As these events continuously flow they need to be identified, have a response very quickly and are often "transported" only in memory, so using a database is not a recommended design option.
• Event stream
• Events arrive for processing in an event stream or streams and they are time ordered.
• Event type
• Event classification which defines the properties (i.e., name-value pairs) associated with the event.
• Event Processing Network (EPN)
• The EPN (aka Directed Flow Graph) is designed as a loosely-coupled collection of event nodes, each performing a unique action on the events as they pass through the network.
• Event nodes could be one of the following types:
• Event adapter
• Event channel
• Event cache and event POJO Bean node
• Event processor
• Filtering, correlation, aggregation, or pattern matching can be applied

To facilitate event processing, an in-memory extensible event technology built on EDA has been provided by Oracle, which is introduced in this book:

The Book[6]

This book is aimed for both developers and architects that need to learn about event processing, stream processing, and the event-driven architecture.  It uses tutorials and examples to help you create event-driven applications using Oracle CEP.  Finally, it covers some of the above components seen in the diagram in details.

References

1. Luckham, David C. (2012). Event Processing for Business: Organizing the Real-Time Enterprise. Hoboken, New Jersey: John Wiley & Sons, Inc.,. p. 3.
2. Schmerken, Ivy (May 15, 2008), Deciphering the Myths Around Complex Event Processing, Wall Street & Technology
3. Bates, John, John Bates of Progress explains how complex event processing works and how it can simplify the use of algorithms for finding and capturing trading opportunities, Fix Global Trading, retrieved May 14, 2012
4. Complex event processing (Wikipedia)
5. Event-driven architecture (Wikipedia)
6. Getting Started with Oracle Event Processing 11g (Reviewed book)
7. All Book Reviews on XML and More

Analyze Hanging Programs Using Java Thread Traces

In this article, we will discuss the following topics:

• How to analyze hanging, deadlocked or frozen programs?
• How to generate stack traces?
• What is deadlock?

What to Do When Your Applications Hang?

If you think your program is hanging, generate a stack trace.  A stack trace of all threads can be useful when trying to diagnose a number of issues such as deadlocks or hangs.

When looking at stack traces, check the following:

• If a thread waits on
• A monitor lock
• A condition variable
• If system threads show up as the current threads
• If the program is deadlocked then some of the system threads will probably show up as the current threads, because there is nothing else for the JVM to do

How to Generate Stack Traces?

With thread stack trace, you can analyze the source of deadlocks.  There are multiple approaches:

• Using jstack command, it can 
• attach to the specified process (or core file) and prints the stack traces of all threads that are attached to the virtual machine (this includes Java threads and VM internal threads).
• For each Java frame, the full class name, method name, 'bci' (byte code index) and line number, if available, are printed.
• Obtain stack traces from a core dump:
• jstack $JAVA_HOME/bin/java core
• Be used to print a mixed stack.  That is, it can print native stack frames in addition to the java stack. 
• • Native frames are the C/C++ frames associated with VM code, and JNI/native code.
  • To print a mixed stack the -m option is used.
• When a deadlock occurs, doing a Ctrl + Break on Windows forces a Java level thread stack trace to print to standard output. On Solaris and Linux, sending a SIGQUIT signal to the Java process id does the same.
• Beginning with Java 6, the bundled JConsole tool added the capability to attach to a hung Java process and analyze the root cause of the deadlock.

What Is Deadlock?

If you're working with a moderately complex multithreaded program, then sooner or later, you'll hit the problem of deadlock^[2,3].  Deadlock is the phenomenon when, typically, two threads each hold an exclusive lock that the other thread needs in order to continue.  In principle, there could actually be more threads and locks involved. Most of the time, a deadlock is caused by acquiring locks in the wrong order.

Deadlock can occur with any locking primitive. It notably occurs with the synchronized keyword, but it's liable to occur with locks, Semaphores, blocking queues etc.

A Deadlock Example

We used jstack to generate Java thread traces as follows:
$ jstack 3554

Our VM is HotSpot Client from JDK 7:

Found one Java-level deadlock:
=============================
"RunLevelControllerThread-1377458723460":
  waiting to lock monitor 0x871756a8 (object 0x93480480, a java.util.logging.LogManager$LoggerContext),
  which is held by "RunLevelControllerThread-1388458723457"
"RunLevelControllerThread-1388458723457":
  waiting to lock monitor 0x87173448 (object 0x93472290, a java.util.logging.LogManager),
  which is held by "RunLevelControllerThread-1377458723460"

Java stack information for the threads listed above:
===================================================
"RunLevelControllerThread-1377458723460":
    at java.util.logging.LogManager$LoggerContext.findLogger(LogManager.java:489)
    - waiting to lock <0x93480480> (a java.util.logging.LogManager$LoggerContext)
    at java.util.logging.LogManager.getLogger(LogManager.java:910)
    at com.sun.enterprise.server.logging.LogManagerService.postConstruct(LogManagerService.java:412)
    - locked < x93472290> (a java.util.logging.LogManager)
    - locked <0x93465158> (a java.lang.Class for java.util.logging.Logger)


"RunLevelControllerThread-1388458723457":
    at java.util.logging.LogManager.drainLoggerRefQueueBounded(LogManager.java:811)
    - waiting to lock <0x93472290> (a java.util.logging.LogManager)
    at java.util.logging.LogManager$LoggerContext.addLocalLogger(LogManager.java:511)
    - locked < x93480480> (a java.util.logging.LogManager$LoggerContext)

In the above example, you can see that two threads:

• RunLevelControllerThread-1377458723460 
• RunLevelControllerThread-1388458723457 

were waiting for locks that were held by each other.  This have created a deadlock.

References

1. jstack - Stack Trace
2. Deadlock
3. How to Avoid Deadlocks (Xml and More)