BT

InfoQ Homepage News Greg Young on Using Complex Event Processing

Greg Young on Using Complex Event Processing

Bookmarks

For problems that have to do with time like querying over historical data Complex Event Processing, CEP, can be very useful, Greg Young suggests in a recent presentation.
Greg describes event processing as a method of analysing streams of data about things that has happened and deriving a conclusion from them. One example is temporal correlation queries used when you want to correlate things that have happened at different times, e.g. finding users that said the word “presents” within one minute of saying the word “birthday” in a twitter stream or finding patients in a clinical trial that has reacted in certain ways at different times during the trial period.

For creating queries Greg uses a query language implemented in JavaScript that works on top of event streams. Writing queries in JavaScript means a web browser can be used for writing, and also for debugging.
This query language is included in Event Store, a type of NoSQL database built according to the concepts of an Event Source. In event sourcing, instead of representing state directly, state is represented as a series of facts, something that has happened at a specific point in time. Storing data this way means that no information is lost since every change of state is stored, not just the current state.

Storing facts or events creates a kind of time machine making it possible to go back in time and look at how things were historically, e.g. creating a new report today and run it as if time is one month back.
Another use case is looking at predictions and estimating the predictivity capability; running a predictive model as if it is one month earlier, predicting outcome for today and comparing the result with the real outcome.

Event Store is an open source product, licensed under a 3-clause BSD license, first released in September 2012.

Greg Young, lead architect for the Event Store, is an independent consultant famous for having coined the term CQRS.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Still too much mis-understanding of event processing

    by peter lin /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    The talk is interesting, but it's rehashing a lot of common mis-understanding of event processing. Event processing was never about "now". Just because some CEP products sold that as "the story", the history of event processing goes back at least 4 decades. The way Rich Hickey explains Datomic is a better explanation of this "class of problems" with grounding in formal logic.

  • Re: Still too much mis-understanding of event processing

    by William Louth /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Peter I'd argue that CEP is probably past it's "sell by" date. Unfortunately "Complex" never resulted in the complexity that exists within living systems. There was complexity but it was not the complexity many had hoped to achieve (or emerge) not to deal with. Event streaming which I suspect is how many now view CEP is still a means to an end and that end I believe is simulation (simulated playback) of the domain or the universe (if future predictions are accurate...note the absence of a timeline ;-) ).

    The power of playback simulation (based to some degree on discrete event simulation) is learning through experience

    www.jinspired.com/site/simulation-and-time-sync...

    the altering of space and time

    www.jinspired.com/site/changing-space-and-time-...

    and finally the ability to move the intelligence (that is still lacking) to a machine mind whose existence is not tied to the lifetime of its body parts which you could view as applications.

    www.jinspired.com/site/mirroring-mindreading-an...

  • Re: Still too much mis-understanding of event processing

    by peter lin /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Agree 100% CEP is past "sell by" date. It wasn't "new" when it came out, just the sales hype tried to paint it as new. Most of the first generation CEP couldn't really be used for what you call "simulated playback" where different scenarios are used to try out different "what if" scenarios. They were only really good for literal playback of a sequence of events, though even then the issue of partially-ordered versus totally-ordered sets reared it's ugly head.

    The biggest limitation is that most of the CEP products early on only covered sliding windows. They ignored 95% of the challenges related to temporal logic and temporal database. At some point, people want to playback the events with different models and scenarios, but most of the products didn't care about saving those events to a temporal database. It was up to the developer to figure out how to efficiently store temporal data. It's understandable most of the products didn't tackle temporal database problem, since "the field" made very little progress after tsql2 was rejected.

    Now that the hype around CEP has died down, hopefully more fruitful progress can be made towards solving the broader problem of how to reason over temporal data and how to efficiently store/query it.

  • Re: Still too much mis-understanding of event processing

    by Greg Young /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    << Greg Young here

    I think you might have misunderstood the talk or maybe in time constraints it was not explained well enough the query language is over HISTORY that transparently moves into now.

    Greg

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT

Is your profile up-to-date? Please take a moment to review and update.

Note: If updating/changing your email, a validation request will be sent

Company name:
Company role:
Company size:
Country/Zone:
State/Province/Region:
You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.