BT

Focus on the Process, Not on Individual Microservices

by Jan Stenberg on Dec 18, 2016 |

The key to success when working with a microservices based distributed system is to focus on the distributed process as a whole, not on the microservices themselves. The services are the least important part, Eric Ess claimed at the recent Microservices Conference in London, in his presentation on how to monitor distributed processes at jet.com.

At Jet a process initiated by a user involves at least a few microservices to complete and is called a distributed process; Ess, Director of Engineering, explains and notes that this is a key term for them when looking at how their system execute user requests.

Jet has about 800 microservices in production today giving a very complex communication topology. Because of this complexity, it’s infeasible for any team to know what’s happening outside of their scope, as well as being impossible for any individual to fully understand the system architecture. Despite this complexity, during problems in production, it’s essential to know exactly what the root cause is, and in which service it originates.

To overcome this challenge, there are two key things they want to accomplish:

  • Know how a single process is behaving, what microservices it has passed through and what it’s currently doing- basically be able to follow different type of processes as they move through the system via microservices interacting with each other.
  • Validate processes by defining the expected workflow for a given process and then validate that it follows that path when executed. Ess notes that even though a process is not generating any errors, it can still be behaving incorrectly. One example is a bug in A/B testing that routes a process the wrong way, causing a flaw in the testing data.

Ess notes that by focusing on the distributed process as a whole, not on the microservices themselves, they can ignore the services; they are a means for moving the process to the next service and a step towards process completion. The current state of the process and what is happening to it is what they care about.

This requires an altered mindset, with engineers focusing on the behaviour of a process within the system, not on a microservice and how it should behave when receiving a message. A team is not building individual microservices, but microservices that interacts with other services around it.

There are a lot of tools available to evaluate microservices or a system, but not to evaluate the process, or the behaviour of the process, as it’s being executed. In addition, Jet is using F# and since it’s hard to find suitable tools targeting F# they have created their own toolbox.

To provide a view of the running system and its processes, they have created a communication protocol (Dr Orpheus) which provides a set of header metadata that goes into every message and some rules for what a microservice must do when receiving a message with metadata. They are also building a telemetry processing / data streaming engine (XRay) that is doing some basic complex event processing (CEP), collecting data that every microservice emits as it processes messages. Engineers and business people can now supervise all processes and react when they are misbehaving in any way, not following the predefined flow, progressing too slowly or blocking in some service.

Next year’s Microservices Conference in London is scheduled for November 6-7, 2017.

Rate this Article

Relevance
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss
General Feedback
Bugs
Advertising
Editorial
Marketing
InfoQ.com and all content copyright © 2006-2016 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.