Event processing is a method of tracking and analyzing (processing) streams of information (data) about things that happen (events), and deriving a conclusion from them. Complex event processing, or CEP, is event processing that combines data from multiple sources to infer events or patterns that suggest more complicated circumstances. The goal of complex event processing is to identify meaningful events (such as opportunities or threats) and respond to them as quickly as possible.
Using airport as an example, events happening in the airport that may require tracking include missing luggages, passenger movements, etc. An architecture combining Event Driven Architecture (EDA) and Service Oriented Architecture (SOA) can be deployed for such purpose. A system utilizing Oracle products is illustrated in the following diagram:
In this book, it has covered the following topics and more:
- Product history of Oracle CEP
- Event Driven Architecture
- Requirements of Complex Event Processing
- Creating Event Driven Architecture (EDA) applications using Oracle CEP
History of Oracle Event Processing Solution 11gR1
A product initially named Elvis (or WebLogic Event Server) was developed by the Time and Event Driven (TED) team in early 2007 at BEA Systems Inc. After several "man years" of effort, this technology was first released on July 16, 2007 from BEA Systems.
In parallel, a Complex Event Processing "engine" was researched and developed separately within Oracle. Starting from 2008, WebLogic Event Server and this CEP "engine" were integrated. Finally, on July 1, 2009, these combined technologies were released as the Oracle Event Processing Solution 11gR1. This product is claimed to be one of the most complete and powerful in-memory extensible event analytics technologies available in the market.
Limitations of Traditional Systems for Event Processing
After the emerging of social media, data are continuously created in vast quantities. This proliferation of data (or Big Data grow) has brought challenges to traditional systems for tracking and analyzing streams of information (data) about things that happen (events).
First thought on supporting events is using traditional systems. However, they have their limitations:
- Traditional computing systems are mostly based on database
- Information must be inserted and statically placed. Only after that, can users analyze and respond.
- Frequent disk reads and writes will quickly become system's performance bottlenecks.
- Traditional JEE Application Servers adopt the send/reply implementation paradigms
- This has introduced delays that won't meet the real-time requirements needed for event processing.
So, a new Event Driven Architecture (EDA) was introduced, which processes event streams in memory using a defined window of time or other filtering options.
Event-Driven (Architecture) Based Systems
Event-driven architecture (EDA) is a software architecture pattern promoting the production, detection, consumption of, and reaction to events. Before systems can process any event, they need to normalize the incoming data stream by converting them into events and event types. These representations are then processed downstream or upstream in the Event Processing Network (EPN).
Before proceeding further, let's introduce you the following definitions:
- In the context of this book, an event is an object that has a change in its state immediately, or over a period of time.
- As these events continuously flow they need to be identified, have a response very quickly and are often "transported" only in memory, so using a database is not a recommended design option.
- Event stream
- Events arrive for processing in an event stream or streams and they are time ordered.
- Event type
- Event classification which defines the properties (i.e., name-value pairs) associated with the event.
- Event Processing Network (EPN)
- The EPN (aka Directed Flow Graph) is designed as a loosely-coupled collection of event nodes, each performing a unique action on the events as they pass through the network.
- Event nodes could be one of the following types:
- Event adapter
- Event channel
- Event cache and event POJO Bean node
- Event processor
- Filtering, correlation, aggregation, or pattern matching can be applied
This book is aimed for both developers and architects that need to learn about event processing, stream processing, and the event-driven architecture. It uses tutorials and examples to help you create event-driven applications using Oracle CEP. Finally, it covers some of the above components seen in the diagram in details.
- Luckham, David C. (2012). Event Processing for Business: Organizing the Real-Time Enterprise. Hoboken, New Jersey: John Wiley & Sons, Inc.,. p. 3.
- Schmerken, Ivy (May 15, 2008), Deciphering the Myths Around Complex Event Processing, Wall Street & Technology
- Bates, John, John Bates of Progress explains how complex event processing works and how it can simplify the use of algorithms for finding and capturing trading opportunities, Fix Global Trading, retrieved May 14, 2012
- Complex event processing (Wikipedia)
- Event-driven architecture (Wikipedia)
- Getting Started with Oracle Event Processing 11g (Reviewed book)
- All Book Reviews on XML and More