BT

Your opinion matters! Please fill in the InfoQ Survey!

RockScript: An Open Source Scripting Language and Engine for Microservice Orchestration

| by Daniel Bryant Follow 200 Followers on Oct 27, 2017. Estimated reading time: 5 minutes |

Tom Baeyens has released a preview of RockScript, an open source scripting language and engine for integrating and orchestrating microservices in the style of Event-Driven Architecture (EDA). RockScript uses a language that looks much like JavaScript, which when executed by the accompanying engine allows the coordination of transactional activities within microservice systems in a similar fashion to Business Process Execution Language (BPEL) and Business Process Modelling (BPM).

The challenge of coordinating service interaction within a distributed system is not a new issue, and the choice between "orchestration and choreography" has been discussed extensively within the traditional Service Oriented Architecture (SOA) community. Service orchestration refers to the automated execution of services on a pre-defined business process workflow by a centralised engine (for example using BPEL). Service choreography generally involves no centralised governance for business processes, and instead the coordination of service interaction is built into the services themselves; this has been referred to as "smart endpoints and dumb pipes" by microservice luminaries James Lewis and Martin Fowler.

The discussion of orchestration and choreography has continued within the microservice community, with the generally accepted best practice in this architectural style being the choreographing of services. However, the increasing number of enterprise organisations that are adopting the microservice architectural style (and have significant existing investment in orchestrated process), in combination with the rise in popularity of fine-grained services being implemented using serverless / Function-as-a-Service (FaaS) frameworks, has resulted in increased focus once again on orchestration frameworks. RockScript is one example of this type of orchestration framework that is specifically targeting microservice implementations.

The RockScript wiki states that "interactions with microservices are mostly done in JSON over HTTP", and therefore these interactions cannot easily participate in a transaction. At any point during an interaction the service being called may crash, or some component of the underlying infrastructure fail. In order to keep consistency when combining multiple microservice interactions for a business process, a record of the call graph and completions and failures must be kept and managed. A microservice best practice example of this type of multi-phase transaction protocol is captured within the Saga Pattern.

The RockScripts language is designed to capture what must happen in response to an event. Typically an external event is used as the input trigger, and this results in commands being sent to other microservice APIs, potentially with data transformations in between. RockScript is designed to be connected with both an HTTP API and a message exchange, although currently only the HTTP API is implemented. Baeyens, the project creator and founder of the Activiti and jBPM platforms, stated that the vision of RockScript is "to make it easy to write small scripts that are connected to events being fired in enterprise systems, and we want to make it easy to establish those type of trigger connections to scripts." This is similar to the concept of triggers in the workflow automation frameworks IFTTT or Zapier.

RockScript executions are resilient, and the runtime engine ensures that script executions can recover from application and infrastructure crashes. The runtime state of each script execution is stored across a cluster using an event sourcing approach, and therefore script executions don't consume resources like threads and memory when they are waiting for asynchronous activities to complete. The use of event sourcing also allows the inspection of what happened when something goes wrong, such as a server crash, and it is possible to attempt recovery from the last saved execution position. 

InfoQ sat down with Baeyens and discussed his motivation for creating RockScript.

InfoQ: Welcome to InfoQ Tom! You have been involved workflow and Business Process Management (BPM) for a long time (having founded jBPM, Activiti and Effektif in the past), and so why did you start this new project?

Baeyens: Those projects gave me a ton of background in implementing engines that deal with asynchronous continuations. But on the product side, I realized that developers had to work in 2 different worlds: Workflow are based on a diagram. Data transformations had to be put in between the activities which were typically written in JavaScript or some other programming language.

Developers then had to work across these two distinct worlds. I believe that is an important reason why workflow did not yet breakthrough as a mainstream developer technology. By bringing the workflow activities concept to a scripting language like JavaScript developers can can work in the single world which they like. My ambition is to bring microservice integration to a much broader developer community and make integrating microservices a lot easier.

InfoQ: What are other technologies that are comparable with RockScript?

Baeyens: Recently cloud platform vendors launched similar services like AWS step functions and Microsoft logic apps. These solutions present themselves as the glue between API's, custom apps and serverless cloud functions (aka lambda's). Service interactions nowadays are typically based on JSON, with transport over HTTP and shifting towards asynchronous messaging. Historically BPEL aimed to address integration XML based services although that attempt was horrendously complex. But it shows that the need for an integration programming language is constant. With RockScript, the focus is to be the integration script language that fits best with today's API and microservice ecosystem.

InfoQ: We note that you have a strong pedigree in Java, but have chosen JavaScript as the basis for RockScript. Why was this?

Baeyens: There are 3 reasons for it:

  1. Because I envision a world where complete applications can be build on serverless cloud infrastructure with online IDE's and only JavaScript knowledge.
  2. It's important for CTO's and dev team leads to know that developers will be up and running in no time and that new developers will be quick to read and learn the technologies adopted by their company. Because RockScript is a subset of JavaScript, it will be very easy for most developers to learn it.
  3. JavaScript syntax is also ideal to perform the data transformations in between the microservice interactions. Most microservices are based on JSON and JavaScript is the best language to manipulate that type of data.

The RockScript wiki states that the project is currently at proof-of-concept stage, and as the implementation is volatile no stability guarantees are offered at this point in time. The current implementation is focused on developer experience, and as such persistence is currently offered only in-memory (with a plan to build persistence to a datastore like SQL or Kafka), and coverage of the JavaScript language is still minimal. Feedback is being sought, and developers can interact with the project via GitHub issues, Twitter, or mailing list.

Several new service orchestration frameworks are emerging in this space, and interested readers can find additional details on each project's website:

Existing integration frameworks, such as Apache Camel and Spring Integration, can also be used to emulate much of the orchestration behaviour. There is also similarity shared with existing Business Process Modelling (BPM) tooling, such as Activiti, jBPM, and MS BizTalk BPM.

Additional information on RockScript can be found on the project's website and GitHub repository, and a hands-on tutorial can be found on the project Wiki.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Look at AWS Simple Workflow (SWF) and Uber Cadence by Maxim Fateev

AWS Simple Workflow Service and its open source analog Uber Cadence are production ready examples of orchestration services that use code for microservice orchestration.

Re: Look at AWS Simple Workflow (SWF) and Uber Cadence by Richard Clayton

I don't know anything about RockScript, but I'm guessing this is an analog to AWS Step Functions. We already see a lot of community effort developing platform agnostic Lambda engines -- my guess is that this is the next logical step (having the orchestration engine).

Re: Look at AWS Simple Workflow (SWF) and Uber Cadence by Tom Baeyens

@Maxim: While AWS Simple Workflow Service, AWS Step Functions and Uber Cadence are alternatives and target the same use cases, their approach is a fundamentally different.

In RockScript, we take the perspective of the developer. The developer can write a script as if it contains blocking code (the activities). The engine executes the activities non-blocking. You can write eg
for (var order in orders) {
myService.myLongRunningActivity(order);
}

AWS Step Functions and Uber Cadence are developed from the engine's perspective. The engine will do a callback to your code when an activity is done and your code needs to calculate which activity is next. A workflow corresponds to a collection of callbacks. It's much harder for the developer and forcing a non intuitive development style imo.

In those solutions you'll have to implement a callback like this
NextActivity whatsNextAfterMyActivityIsDone(WorkflowContext wfc);

You'll have to use the context to keep track of which order was processed. If there is a next one, return the next myActivity otherwise you have to return the next activity. This callback style is much harder to write and maintain imo.

And if you need to do data transformations between two activity invocations, it's much easier to understand that logic if you see both of those invocations and the transformation code inbetween. Instead of cutting those things up in pieces in different callback handlers.

Re: Look at AWS Simple Workflow (SWF) and Uber Cadence by Maxim Fateev

While SWF and Cadence service API is callback based its client model is exactly the one that you described. For example Cadence Go library provides blocking Activity invocation semantic. So you sample looks like:

for index, order := range ordrers {
// ExecuteActivity returns Future. Future.Get blocks until activity is completed.
var activityResult ActivityResult
err := cadence.ExecuteActivity(ctx, myLongRunningActivity, order).Get(ctx, &activityResult);
}

It also supports
cadence.Sleep(..);
which blocks for any amount of time, but in reality doing it asynchronously without consuming any worker resources.

See github.com/samarabbas/cadence-samples for more real life executable samples.

I believe that inventing a new language is not the best approach as the Cadence model can work with any language including JavaScript or TypeScript if you want a blocking semantic.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

4 Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT