News

Complex Event Processing and EDA?

Posted by Steven Robbins on Aug 21, 2008 06:58 AM

Community: Architecture,; SOA
Topics: Composition ,; Loose Coupling
Tags: CEP ,; Service Component Architecture ,; Event Stream Processing ,; SOA Adoption

Complex Event Processing (CEP) systems and Event Driven Architectures (EDA) have been identified as playing a larger role in sophisticated systems today and in the future. What that role is and how it is carried out are up for debate. David Luckham and Roy Schulte put together an overview and glossary of terms used in CEP and EDA.

Luckham and Schulte defined each term independent from implementation specifics saying:

The most basic term, event, is problematic. Essentially there are two distinct meanings; (1) an activity that happens, and (2) something that represents that activity in a computer system. It is tempting to introduce two separate terms such as “event” and “event object“. However, in any discussion longer than a paragraph or two this becomes intolerably clumsy and one finds the distinction being misused, forgotten or dropped altogether. For example, using the two separate terms would dictate that ”event processing” (see below) should be “event object processing“. The best solution is to overload the word “event“. The context of each use becomes the indicator of which meaning is intended.

Luckham and Shulte defined complex event as "an event that is an abstraction of other events called its members." Complex events were the topic of Joe McKendrick's choice when he cited a problem with a system at Moody's Investors Service that caused incorrect ratings to be assigned. McKendrick said "The implication is that millions, if not hundreds of millions, of dollars in investment decisions may have been made on faulty data generated by the system." McKendrick's position was that complex sense and respond systems may still require human involvement to prevent problems or errors.

McKendrick pointed out that Dr K. Mani Chandy, a Computer Science professor from California Institute of Technology doing research around Sense and Respond (S&R) systems, talked about keeping a human in the loop when the results of the decision made based on complex events. Chandy said that in situations like tactical military operations where the decision may be to use a weapon "It’s always done with a human being responsible for their final action."

Chandy along with Micahel Olson talked about how event processing and 'sense and respond' applications (PDF) may become ubiquitous in the area of Business Activity Monitoring and business dashboards. Chandy and Olson dug into the area of web S&R applications saying that these are S&R applications that get events and data from web data sources:

Web data sources can be active or passive. A client can poll a server, using a request- reply protocol to extract information. Alternatively, information can be pushed to a client by means of RSS or ATOM streams or by using other data protocols. A passive data source can be made to have an active interface by an agent that polls it periodically and actively sends information about changes in successive polls.

But does CEP require a completely different type of architecture?

Brenda Michelson gave an overview of Event Driven Architecture in the context of event processing. Michelson identified 5 categories of components in an EDA:

Event Metadata: Event metadata includes event specifications and event processing rules.
Event Processing: The cores of event processing are the engine and the event occurrence data.
Event Tooling: Event development tools are required to define event specifications and processing rules, and to manage subscriptions. Event management tools provide administration and monitoring of the event processing infrastructure, monitoring of event flows, and visibility into event generation and processing statistics.
Enterprise Integration: An enterprise integration backbone plays a large role in event-driven architecture. Some of the required integration services are: event preprocessing (filters, routes, and transformations), event channel transport, service invocation, business process invocation, publication and subscription, and enterprise information access.
Sources and Targets: The resources of the enterprise—applications, services, business processes, data stores, people, and automated agents—that generate events and/or perform an event-driven action.

Michelson also discussed the relationship between EDA and SOA:

I believe that SOA and EDA are peers and complements. So, I disagree with the SOA evangelists who say EDA is merely a subset (child) of SOA. A broader event-driven architecture stretches beyond event-driven SOA, to include real-time information flow and analysis, and complex event processing.

Dr Ivar Jacobson took a different position on EDA. Jacobson asked the question "Do we need event-driven architecture?" In answering his own question, Jacobson that, while EDA puts the focus on the events as the most important element in the system, you are better of putting your focus on "components or services and some kinds of "channels" between some of these components." Events can be produced, delivered, consumed, and even broadcast in a system like this. One big benefits of this type of system is that

The most interesting components are services. You get service-oriented architecture (SOA) at the same time, and more.

In either case, EDA and SOA do not preclude or exclude each other. They can be used to provide complex event processing systems that can provide automatic or actionable outcomes for your enterprise.