BT

Your opinion matters! Please fill in the InfoQ Survey!

Event Sourcing in an Unreliable World

| by Jan Stenberg Follow 9 Followers on Nov 30, 2017. Estimated reading time: 2 minutes |

A note to our readers: As per your request we have developed a set of features that allow you to reduce the noise, while not losing sight of anything that is important. Get email and web notifications by choosing the topics you are interested in.

Examples of event sourced systems are commonly from domains like e-commerce that are process-oriented with incoming commands that generate events, and where we are in control of the process. But there are domains without processes, where we are collecting external events; domains that are intrinsically unreliable with event sources and transports that are unreliable, Lorenzo Nicora explained at the recent Microservices Conference µCon London 2017. Examples include logistics and transport domains as well as IoT and mobile apps that depend on an unreliable network and may be temporarily disconnected.

Nicora, platform engineer at Buildit@wipro digital, noted that in these unreliable domains things happens in the external world that are out of our control; events can be missing or arrive out of order. We just have to gather events as they arrive and afterwards try to draw a picture of the external world that makes sense. Essentially, we have to drop the distinction between commands and events that we see in process-oriented domains, and start to accept all events coming in, validate and store them as fast as possible, and after that apply logic to build the read models. Nicora believes this to be the same type of mind shift as when moving from ACID to BASE in the data storage world, and he calls this approach:

Write Fast, Think Later.

Validating events in this scenario is mainly a security check to prevent the system from forged or malicious events and is completely stateless. It is therefore possible to run both validation and storing of events in parallel, which is great from a scalability perspective. Using a microservices architecture, separate services can process writes of events, build read models and serve queries which gives a very scalable solution since every service is independently scalable.

Advantages when storing all events in these unreliable domains include the possibility to:

  • Reorder events if events are arriving late and out of order
  • Guess missing events
  • Rebuild read models if you find inconsistencies in models or when late events are received
  • Delay read models to give events time to arrive

When ordering events, the only meaningful timestamp on an event is the time it was created in the source. The problem is that this timestamp is only reliable per device emitting events, which means we have no reliable global event order, and there is no easy way around this. If ordering between events from different devices is important, we have to deal with that in logic after the events are stored. Nicora also mentions that with this design the read models will always be delayed; how much depends on the domain and the implementation.

Nicora concluded by noting that if you are working in an unreliable world with high scalability requirements, then this weak write consistency model may be a good fit. He emphasized though that when using it, we should not try to make sense of events on write; instead, we should focus on building consistent read models.

Next year’s conference is scheduled for November 5-6, 2018.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT