Greg Young on Using Complex Event Processing

| by Jan Stenberg Follow 38 Followers on Jan 29, 2014. Estimated reading time: 1 minute |

For problems that have to do with time like querying over historical data Complex Event Processing, CEP, can be very useful, Greg Young suggests in a recent presentation.
Greg describes event processing as a method of analysing streams of data about things that has happened and deriving a conclusion from them. One example is temporal correlation queries used when you want to correlate things that have happened at different times, e.g. finding users that said the word “presents” within one minute of saying the word “birthday” in a twitter stream or finding patients in a clinical trial that has reacted in certain ways at different times during the trial period.

For creating queries Greg uses a query language implemented in JavaScript that works on top of event streams. Writing queries in JavaScript means a web browser can be used for writing, and also for debugging.
This query language is included in Event Store, a type of NoSQL database built according to the concepts of an Event Source. In event sourcing, instead of representing state directly, state is represented as a series of facts, something that has happened at a specific point in time. Storing data this way means that no information is lost since every change of state is stored, not just the current state.

Storing facts or events creates a kind of time machine making it possible to go back in time and look at how things were historically, e.g. creating a new report today and run it as if time is one month back.
Another use case is looking at predictions and estimating the predictivity capability; running a predictive model as if it is one month earlier, predicting outcome for today and comparing the result with the real outcome.

Event Store is an open source product, licensed under a 3-clause BSD license, first released in September 2012.

Greg Young, lead architect for the Event Store, is an independent consultant famous for having coined the term CQRS.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Still too much mis-understanding of event processing by peter lin

The talk is interesting, but it's rehashing a lot of common mis-understanding of event processing. Event processing was never about "now". Just because some CEP products sold that as "the story", the history of event processing goes back at least 4 decades. The way Rich Hickey explains Datomic is a better explanation of this "class of problems" with grounding in formal logic.

Re: Still too much mis-understanding of event processing by William Louth

Peter I'd argue that CEP is probably past it's "sell by" date. Unfortunately "Complex" never resulted in the complexity that exists within living systems. There was complexity but it was not the complexity many had hoped to achieve (or emerge) not to deal with. Event streaming which I suspect is how many now view CEP is still a means to an end and that end I believe is simulation (simulated playback) of the domain or the universe (if future predictions are accurate...note the absence of a timeline ;-) ).

The power of playback simulation (based to some degree on discrete event simulation) is learning through experience

the altering of space and time

and finally the ability to move the intelligence (that is still lacking) to a machine mind whose existence is not tied to the lifetime of its body parts which you could view as applications.

Re: Still too much mis-understanding of event processing by peter lin

Agree 100% CEP is past "sell by" date. It wasn't "new" when it came out, just the sales hype tried to paint it as new. Most of the first generation CEP couldn't really be used for what you call "simulated playback" where different scenarios are used to try out different "what if" scenarios. They were only really good for literal playback of a sequence of events, though even then the issue of partially-ordered versus totally-ordered sets reared it's ugly head.

The biggest limitation is that most of the CEP products early on only covered sliding windows. They ignored 95% of the challenges related to temporal logic and temporal database. At some point, people want to playback the events with different models and scenarios, but most of the products didn't care about saving those events to a temporal database. It was up to the developer to figure out how to efficiently store temporal data. It's understandable most of the products didn't tackle temporal database problem, since "the field" made very little progress after tsql2 was rejected.

Now that the hype around CEP has died down, hopefully more fruitful progress can be made towards solving the broader problem of how to reason over temporal data and how to efficiently store/query it.

Re: Still too much mis-understanding of event processing by Greg Young

<< Greg Young here

I think you might have misunderstood the talk or maybe in time constraints it was not explained well enough the query language is over HISTORY that transparently moves into now.


Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

4 Discuss