An Introduction to Apache ODE

Posted by Paul Brown on Sep 03, 2007 11:01 AM

Community

SOA

Topics

Business Process Management

Tags

WS-BPEL,

WS-Star,

BPEL

Apache ODE ("Orchestation Director Engine") aims to produce an implementation of the WS-BPEL 2.0 ("Web Services Business Process Execution Language") standard suitable for embedding in a generic runtime context. ODE recently graduated from incubation to a fully-fledged top-level project and had its first release since leaving incubation. This note provides a nickel tour and teaser for ODE along with some BPEL concepts in the form of deploying and executing a simple process.

The ODE philosophy on BPEL is that it is a language for describing how to implement a set of message-based communication capabilities in terms of state manipulation and messages exchanged with external services. Other than in this sentence and in the preceding paragraph, the word "business" will not appear, and there will be no talk of alignment with IT or other silliness — ODE is guilt-free (and gilt-free) technology like a web server or a database; what you do with it is up to you. No GUI, IDE, ESB, or other TLA (other than a little XML) is required. Implementing an orchestration engine is a tantalizing but daunting task, and ODE encapsulates the details of concurrency, durable continuation (also called dehydration/rehydration), reliability, and recovery. Perhaps most importantly, ODE is delivered as a component rather than a framework in the hope that it can serve as a baseline for developers looking to add orchestration functionality to their systems.

Where it's needed for clarity, I'll call out a specific version of BPEL, e.g., BPEL4WS 1.1, but otherwise, "BPEL" refers to WS-BPEL 2.0.

Orchestration vs. Everything Else

WS-BPEL 2.0 is part of the WS-* constellation of standards, but for better and worse, it is loosely coupled with the other standards, depending only on WSDL 1.1 and not on SOAP. (The only facet of that minimal coupling that is "worse" is that a number of other standards in domains where BPEL has useful applications, e.g., JBI and WSRF, depend on WSDL 2.0 instead of WSDL 1.1.) As such, a BPEL implementation is orthogonal and complementary to product categories like SOAP stacks, enterprise service buses, integration engines, etc. — just get WSDL parts to the engine and give the engine the infrastructure (e.g., state persistence and timer events) that it needs.

ODE realizes this minimal view through its integration layer abstraction. The ODE core uses an integration layer implementation to receive and deliver messages to external parties and to get access to resources such as threads. ODE includes integration layer implementations for Apache AXIS2 (ODE processes exposed as web services) and ServiceMix (ODE exposed as a JBI service engine), and a number of other implementations are under active development.

WS-BPEL 2.0: The Latest and Greatest (and Only) BPEL Standard

What is now WS-BPEL 2.0 has been almost six years in the making, four of them at OASIS. Here's a rough timeline for the standard and its forebears with links to the CoverPages for some older documents:

WSFL, May 2001 (IBM) (link collection)

The Web Services Flow Language

XLANG, May 2001 (Microsoft) (link collection)

Despite being all-caps, the name is not an acronym.

BPEL 1.0, July 2002 (BEA, IBM, Microsoft) (spec)

A merger of WSFL and XLANG.

BPEL4WS 1.1, March 2003 (BEA, IBM, Microsoft, SAP, Siebel) (link collection)

The specification submitted to OASIS.

WS-BPEL 2.0, March 2007 (OASIS; 39 companies as members of the technical committee) (OASIS TC home page

The first version of the "standard" blessed by a standards organization.

The four years at OASIS did BPEL quite a bit of good, with more than 300 clarifications, corrections, and enhancements thoroughly (if not sometimes too thoroughly) considered and resolved by the technical committee. The four years at OASIS also did BPEL quite a bit of good because many of the companies represented on the technical committee tried to implement or use the language. A point-by-point comparison is between BPEL4WS 1.1 and WS-BPEL 2.0 is probably only of interest to implementors, but here is a short (and non-exhaustive) list of highlights for anyone who ever tried to write a BPEL process by hand:

• Simpler XPath 1.0 syntax for variable manipultation. WS-BPEL 2.0 adopts the XSLT convention of $-prefixed variable names, and if a variable is a WSDL message type, the parts of the message are refered to as $varName.partName. BPEL4WS 1.1 used XPath extension functions like bpws:getVariableData('itemsShipped').
• In-process support for XSLT. WS-BPEL 2.0 includes an XPath extension function doXslTransform(...) for performing XSLT transformations in an assignment operation. BPEL4WS 1.1 did not include any support for in-process transformation, so XSLT operations needed to either be implemented by a proprietary mechanism or as a separate endpoint outside the process.
• Iteration and simple dynamic parallelism. BPEL4WS 1.1 included a while construct for iteration and a flow construct for concurrent enablement of a fixed number of activities, and WS-BPEL 2.0 adds a forEach that iterates over a range of integer values determined at runtime, either simultaneously or in sequence. A parallel forEach makes it possible to directly support dynamic scatter/gather scenarios that were either awkward or impossible in BPEL4WS 1.1.

And the list could go on. The jury is out on just how BPEL processes are likely to get written, whether its via a higher-level notation like BPMN, a graphical notation that maps directly to BPEL, via model-driven means, directly in XML, or in some as yet unknown but closely related non-XML syntax.

Jacob: ODE's Message-Passing Virtual Machine

Two of the more challenging aspects of implementing a BPEL engine are representing the state of an executing process and managing concurrency, and ODE uses a framework called Java Concurrent Objects ("Jacob") to address both. Jacob's approach is a practical combination of ideas from the actor model and process algebra approaches to concurrency and continuation.

Via Jacob, ODE represents the state of a BPEL process as a collection of objects (for lack of a more precise term) connected by channels for passing messages. In this context, a message is a lightweight, internal message – just a Java type – not the heavier WSDL messages that the engine sends and receives externally. When an object receives a message, it can create new channels and objects and send messages; concurrency is managed by consuming messages in a single-threaded fashion (on a per-process-instance basis).

The ODE wiki includes a tutorial on Jacob that covers both motivation and mechanics in terms of BPEL processes, and the Jacob module includes a variety of BPEL-specific and more examples, e.g., an implementation of the Sieve of Eratosthenes. Interested readers can compare the Jacob approach with the approach in Scala.

Getting Going with ODE

For a quick tour of ODE, let's get a simple process deployed and accessible as a web service via the AXIS2 integration layer. The tour is purposefully "low-fi" with only one tiny spritz of doodleware; nothing other than a commandline and an up-to-date JDK (5.0) is required.

Getting ODE and Setting Up

Getting the example running requires the ODE WAR distribution and a relatively recent servlet container; both Tomcat and Jetty are known to work well. The Getting ODE wiki page has a link to a list of mirrors for the ODE 1.1 distribution. Here is a set of instructions for installing ODE with Jetty 6.1.5:

1. Pick a working directory, work for the sake of argument, and unpack the ODE and Jetty ZIP files there. This should create work/apache-ode-war-1.1 and work/jetty-6.1.5 directories.
2. Create an ode directory in work/jetty-6.1.5/webapps.
3. Explode the work/apache-ode-war-1.1/ode.war WAR into the work/jetty-6.1.5/webapps/ode directory.

Or, in Unix commandline form, where "[MIRROR]" should be replaced with the suggested download mirror for ODE:

 $ wget http://[MIRROR]/ode/apache-ode-war-1.1.zip
 $ wget http://dist.codehaus.org/jetty/jetty-6.1.5/jetty-6.1.5.zip
 $ mkdir work && cd work
 $ jar xf ../apache-ode-war-1.1.zip
 $ jar xf ../jetty-6.1.5.zip
 $ mkdir jetty-6.1.5/webapps/ode
 $ cd !$ && jar xf ../../../apache-ode-war-1.1/ode.war

A Simple Example: Multi-Counter Process

As a getting-started example, consider a process that implements a collection of named counters, each with four operations available on a single port:

• {name} -> init -> {} to create a new counter instance.
• {name} -> get -> {int} to access the value of a counter.
• {name} -> getAndIncrement -> {int} to access and increment the value of a counter.
• {name} -> close -> {} to close an existing counter.

BPEL is inherently message-oriented, and I'm using the -> notation above to make sure that request/response isn't confused with RPC. The use of the term "operation" is inherited from WSDL, but its only meaning in a BPEL process is as metadata to use in determining which message-receiving activity (bpel:receive, bpel:pick, etc.) can accept a message.

The distinction between an instance of a process and the process itself merits a digression. In a BPEL engine, a process is a kind of prototype that defines how process instances evolve as messages are sent and received. Depending on the process definition, the BPEL engine is responsible for routing messages to specific instances based on correlation rules or creating new instances to receive messages. For the counter process, each named counter corresponds to a process instance, and the engine determines which instance (if any) should receive a message based on the name of the counter in the message.

Implementing the process in BPEL is straightforward but takes a bit of typing (just under 100 lines, nicely formatted) to produce the XML. Here are the BPEL process counter.bpel and accompanying WSDL counter.wsdl, where in the parts, operations, ports, partner links, and message properties (used to define correlations) are defined.

As discussed above, ODE only uses the abstract portion of the WSDL, i.e., binding and service definitions are ignored. The AXIS2 integration layer, however, will use the binding and service definitions to expose the process as a concrete web service addressable via SOAP and/or HTTP.

For a more condensed version, here is an outline in pseudocode:

receive init(s) and spawn an instance tagged with name = s
 set counter = 0
 thread 1:
 	while (true)
 		receive get(s) for s = name
 		reply with counter
 thread 2:
 	while (true)
 		receive getAndIncrement(s) for s = name
 		reply with counter++
 thread 3:
 	receive close(s) with s = name
 	die // instance only!

(The while(true) construct allows the get and getAndIncrement operations to be invoked arbitrarily many times.)

A few fragments of the BPEL are worth comment. First, the <bpel:import> element at the top of the process defintion tells ODE where to find the accompanying WSDL, i.e., in the same location as the BPEL:

<bpel:import importType="http://schemas.xmlsoap.org/wsdl/"
              location="counter.wsdl"
              namespace="http://example.com/bpel/counter" />

Next, and something's that a default first question on the ODE user mailing list, is the initialization of the counter variable:

<bpel:assign>
   <bpel:copy>
     <bpel:from>0</bpel:from>
     <bpel:to>$counter.value</bpel:to>
   </bpel:copy>
 </bpel:assign>

BPEL requires that variables be initialized prior to use; the engine will generate a bpel:uninitializedVariable fault otherwise.

A graphical representation from the Eclipse BPEL editor helps illustrate the parallel structure of the process:

(The Lomboz 3.3RC1 Eclipse distribution includes integration between ODE and the Eclipse BPEL tooling.)

Checking the Process with bpelc

The ODE distribution includes a commandline wrapper for ODE's BPEL compiler, bpelc. The compiler provides a good sanity check for BPEL processes in the form of static analysis and a full set of cross-reference checks between the process and the imported WSDL and XML Schema. To be precise, the compiler turns raw BPEL XML into a normalized structure that in turn serves as the prototype for the process instances "executed" by the Jacob virtual machine. (The normalization phase takes care of creating any synthetic or implicit constructs needed at runtime, e.g., for compensation.)

For example, to compile the example counter.bpel process with counter.wsdl located in the same directory:

$ export PATH=`pwd`/work/apache-ode-1.1-war/bin:$PATH
$ bpelc ./counter.bpel

If the compiler detects an issue, it will output an error message with line number and other information. Passing a -v or -vv flag will produce more verbose output. As with many languages, not all BPEL processes that compile will run.

The compiler is also addressable programmatically, so it is possible to implement different front-ends that turn a custom language or graphical notation into either XML BPEL or the compiler's intermediate format (called the "O model") and then into the compiled form that the runtime accepts. Activity names, line numbers, and other information are preserved to allow mapping compilation errors or execution traces back to the source format.

Packaging, Deploying, and Executing the Counter Process

Deploying the process in the ODE AXIS2 runtime requires one additional artifact (deploy.xml) that specifies how the BPEL process's partner links map to the ports of a concrete service:

<deploy xmlns="http://www.apache.org/ode/schemas/dd/2007/03"
         xmlns:tns="http://example.com/bpel/counter">
   <process name="tns:counter">
     <active>true</active>
     <provide partnerLink="operations">
       <service name="tns:counter" port="port"/>
     </provide>
   </process>
 </deploy>

In a separate terminal window, start up Jetty:

$ cd work/jetty-6.1.5
 $ ./bin/jetty.sh run

And watch log messages roll by for a minute as it starts up; the last few messages should look something like:

INFO - GeronimoLog.info(79) | Process deployment polling started on path
  /Users/prb/work/jetty-6.1.5/webapps/ode/WEB-INF/processes.
 INFO - GeronimoLog.info(79) | ODE Service Engine has been started.
 2007-08-30 00:59:59.761::WARN:  Unknown realm: Test JAAS Realm
 2007-08-30 00:59:59.830::INFO:  Started SelectChannelConnector @ 0.0.0.0:8080

Create a copy of the deployment directory and move it into place:

$ mkdir /tmp/counter
 $ cp counter.bpel counter.wsdl deploy.xml /tmp/counter
 $ mv /tmp/counter jetty-6.1.5/webapps/ode/WEB-INF/processes

This should produce a bit of logging back in the Jetty window:

DEBUG - GeronimoLog.debug(66) | Created Axis2 service {http://example.com/bpel/counter}counter
 DEBUG - GeronimoLog.debug(66) | Activated {http://example.com/bpel/counter}counter-1 myrole operations: EPR is org.apache.ode.bpel.epr.WSAEndpoint@b56444
 DEBUG - GeronimoLog.debug(66) | Activated {http://example.com/bpel/counter}counter-1
 DEBUG - GeronimoLog.debug(66) | Rehydrating process {http://example.com/bpel/counter}counter-1
 INFO - GeronimoLog.info(79) | Activated process {http://example.com/bpel/counter}counter-1.
 INFO - GeronimoLog.info(79) | Deployment of artifact counter successful: [{http://example.com/bpel/counter}counter-1]
 DEBUG - GeronimoLog.debug(66) | Creating process DAO for {http://example.com/bpel/counter}counter-1 (guid=hqejbhcnphr2jtol7ojc0s)

The configuration of AXIS2 as shipped in the AXIS2 integration layer for Ode supports simple HTTP invocation of operations, so curl is all that's needed to interact with the process from the commandline. (Most *nix systems will include curl, and executables for Windows are available.) This sequence of commands creates a counter named "foo" and increments it twice:

$ curl http://localhost:8080/ode/processes/counter/init?name=foo
 <axis2ns2:initResponse xmlns:axis2ns2="http://example.com/bpel/counter" />
 $ curl http://localhost:8080/ode/processes/counter/get?name=foo
 <axis2ns4:getResponse xmlns:axis2ns4="http://example.com/bpel/counter">
   <value>0.0</value>
 </axis2ns4:getResponse>
 $ curl http://localhost:8080/ode/processes/counter/getAndIncrement?name=foo
 <axis2ns6:getAndIncrementResponse xmlns:axis2ns6="http://example.com/bpel/counter">
   <value>0.0</value>
 </axis2ns6:getAndIncrementResponse>
 $ curl http://localhost:8080/ode/processes/counter/getAndIncrement?name=foo
 <axis2ns8:getAndIncrementResponse xmlns:axis2ns8="http://example.com/bpel/counter">
   <value>1.0</value>
 </axis2ns8:getAndIncrementResponse>

(This isn't RESTful in strict terms, since a GET is creating a resource, but using curl on the commandline beats cobbling together and POSTing SOAP messages!)

A Taste of Management

ODE includes instance and process management functionality addressable via the persistence layer, and in the AXIS2 integration layer, a number of the operations are exposed via web services. For example, to get a list of all deployed processes:

$ curl http://localhost:8080/ode/processes/ProcessManagement/listAllProcesses

Or, if you'd like something more legible, open it in a browser or pipe it through xmllint:

$ curl -s -o - http://localhost:8080/ode/processes/ProcessManagement/listAllProcesses | \
     xmllint --format -
 <ns:process-info-list xmlns:ns="http://www.apache.org/ode/pmapi/types/2006/08/02/">
   <ns:process-info>

     <ns:pid>{http://example.com/bpel/counter}counter-1</ns:pid>
;     <ns:status>ACTIVE</ns:status>
     <ns:version>1</ns:version>
 [...]

And then to create and list a few instances:

$ curl -s http://localhost:8080/ode/processes/counter/init?name=bar
 $ curl -s http://localhost:8080/ode/processes/counter/init?name=baz
 $ curl -s http://localhost:8080/ode/processes/counter/init?name=qux
 $ curl -s -o - http://localhost:8080/ode/processes/InstanceManagement/listAllInstances | \
     xmllint --format -
 <ns:instance-info-list xmlns:ns="http://www.apache.org/ode/pmapi/types/2006/08/02/">
   <ns:instance-info>
     <ns:iid>57</ns:iid>
     <ns:pid>{http://example.com/bpel/counter}counter-1</ns:pid>
     <ns:process-name xmlns:coun="http://example.com/bpel/counter">coun:counter</ns:process-name>
     <ns:root-scope siid="58" status="ACTIVE" name="__PROCESS_SCOPE:counter" modelId="8"/>
 [...]

Next Steps

This just scratches the surface of ODE, but it should reinforce the key takeaways of ODE's philosophical perspective on BPEL and packaging as a component.

ODE is open source, so the next step is to participate:

1. Join the user mailing list and ask/answer questions.
2. Join the developer mailing list and contribute to the direction and implementation of ODE. New integration layers (e.g., SCA), extensions to BPEL, new expression/query languages (e.g., E4X), and enhancements to ODE's static analysis capabilities, or just smoothing out rough edges (e.g., JPA process/instance management) are areas where contributions are welcome.
3. Embed ODE in your project, product, or (software as a) service and then contribute back to the community. ODE was designed and built as an orchestration component rather than a framework, and there are already some commercial success stories for products and (software as a) services built with ODE.

About the Author

Paul Brown is a committer on the ODE project and entrepreneur looking at next big ideas. Paul can be reached at prb@mult.ifario.us.

1. Back to top

   Wrong URL

   Sep 5, 2007 5:35 AM by Odd Christer Brovig

   The url to the article is wrong.

   Reply

2. Back to top

   Re: Wrong URL

   Sep 5, 2007 6:02 AM by Stefan Tilkov

   Thanks - fixed.

   Reply

3. Back to top

   Who's using ODE

   Sep 5, 2007 7:15 AM by Casper Dino

   Any public list of projects using this engine in their solutions? /Dino

   Reply

4. Back to top

   Re: Who's using ODE

   Sep 5, 2007 8:50 AM by Matthieu Riou

   Dino, there's a very early list here: http://ode.apache.org/links.html However from the questions asked on our user and dev mailing lists I can tell you there's quite a few others. We just haven't asked people if we could list their projects there yet.

   Reply

5. Back to top

   Good article

   Sep 8, 2007 8:38 AM by Kristoffer Sjögren

   Hey, good article!

   Reply

6. Back to top

   "close" operation doesn't listed in bindings

   May 9, 2008 11:32 AM by brain debugger

   Hi! Nice article. Looks like the the "close" operation is missing in bindings. Following should be added after "init" operation. Also "while" block for "close" doesn't have statement, don't know if it is ok, though. Regards, Bugger

   Reply

• Older >