BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ
News Articles Presentations Podcasts Guides

Topics

Choose your language

QCon New York

Find real-world practical inspiration from the world’s most innovative software leaders. Attend in-person or online.
QCon San Francisco

Learn what's next in software from world-class leaders pushing the boundaries. Attend in-person or online.
The Software Architects' Newsletter

Your monthly guide to all the topics, technologies and techniques that every professional needs to know about. Subscribe for free.

InfoQ Homepage News Workflow Engine – To Build or Not to Build One?

QCon San Francisco (Oct 2-6, 2023): Find real-world practical inspiration from senior software engineers.

Workflow Engine – To Build or Not to Build One?

Bookmarks

Jul 15, 2009 2 min read

Write for InfoQ

Join a community of experts. Increase your visibility.
Grow your career.Learn more

 

More and more people today realize the importance of introduction workflow into their solutions. But when it comes to the actual implementation, build vs. use, the debate is still going on. In his new post "Workflow engine? Building one..." Bernd Rücker weights in on the issue by discussing several common misconceptions surrounding it.

According to Bernd, typical reasoning for building "home-grown" workflow engines includes the following:

  • We only have very basic requirements, a very simple state machine. A workflow engine is overkill.
  • An engine should be part of our application.
  • We have evaluated the product x and it simply does not fit.

Although, at the first glance, those seem like valid concerns, they rarely justify efforts and expenses of the "home-grown" implementations.

We only have very basic requirements and consequently can’t justify time and effort of learning new technology/implementation. As a result, many implementations start with a simple database table, capturing each process instance and the state it is in. But now, what if it is necessary to support

  • wait states with storing instance variables?
  • timeouts?
  • escalations?
  • decision gateways?

Bernd notes that based on his experience, although implementation does sometimes starts simple, the set of requirements tends to grow over time and at the end companies are typically "stuck" with maintenance and support of the full-blown workflow systems.

An engine should be part of our application and as a result we do not want to introduce a lot of dependences on additional hardware/software/integration/installation procedures (typical requirements for many commercial engines).

Bernd’s suggestion for such scenarios is to consider a lightweight Java workflow engine, which can be easily included into your product. Examples of such engines include, for example, JBoss jBPM, Nova Bonita or Enhydra Shark. This class of engines typically includes a wealth of configuration options, which makes them easily adaptable to specific application requirements.

We have evaluated, it does not fit is one of the hardest arguments to deal with. According to Bernd, the issue here is that a reasonable evaluation, even for a lightweight open source workflow engine, requires time and effort. If not enough time has been spent on evaluation of the engine, then the results are often insufficient. If enough time is spent understanding the technology, it is rare that the implementation does not provide required functionality. Bernd states, for example, that none of his clients found anything, that they could not easily implement using jBPM. It just takes time to understand the technology.

Bernd summarizes his post in a following way:

Please, do not develop! [It is a very expensive undertaking]. The learning curve [for understanding an engine] is almost always worth it!... [Once you understand how to use] the advantage of using an engine [becomes] self-evident.

People today rarely implement their own database or O/R mapper or application server. Why is it often that people think that they should write their own workflow engine? Workflow engines become a commodity and it is nearly always more cost effective to use existing implementation then design and implement your own.

Rate this Article

Adoption
Style

This content is in the Enterprise Architecture topic

Related Topics:

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • workflow engine

    by Winston Dhanraj,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Absolutely agree with Bernd's sumarization, totally relevant for any development effort today.

    I believe this (mal-)practice of home-grown workflow engines is MOST rampant in large Product companies who provide say Telecom, Financial or BSM software. Whatever the compulsions might have been, they could now lead the way by shedding-off their home-grown workflow systems and whole-heartedly embrace open source.

    Back to top
  • why not using a JavaSpace as a lightweight Java workflow engine !?

    by Dominique De Vito,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    First, I agree with Bern's point of view. Workflow use/interest is underestimated.



    Secondly, this may be due to the fact that JavaEE technologies don't support enough workflow implementation. Don't get me wrong: workflows could be implemented using JavaEE technologies, but JavaEE could be better to support workflow; and then, workflow may appear as a beast difficult to master.



    I have advocated JavaSpace interest (saying into my post 7 good reasons to have a second look at Jini, and what is coming next, into the conclusion: "I would like to see a JavaSpace-like included into Java EE"). JavaSpace technology is particulary interesting for building workflow, see another post of mine: JavaSpace into JavaEE: the SemiSpace example.



    What I really like
    with JavaSpace is that it matches
    quite well different use cases.
    I have already mentioned one in
    JavaSpaces,
    an interesting technology for doing ESB or SOA    .
    The workflow use case is another good candidate for
    leveraging JavaSpace technology:


    - the current workflow context could be put into a JavaSpace, as it provides a (simple) storage mechanism,


    - workers could be notified through events
    when context objects into the JavaSpace match a given (state) template,


    - only one worker can take a (context) object into a JavaSpace,
    then
    only one workflow transition could be executed at any time.



    JavaSpace is not a strange beast, it's just the object-oriented JMS cousin !



    So, with JavaSpace included into JavaEE, the line between building its own workflow
    engine, or using an existing one, would be more fuzzy.
    Indeed, JavaSpace would bring quite a big building block so that
    building its own workflow
    engine would be cheaper and less dangerous,
    and changing from a cheap JavaSpace implementation, light-weight workflow,
    to a heavy-weight implementation,
    a more professional implementation (while respecting the spec)
    would be easier too.



    Again I agree with Bern: "Bernd’s suggestion for such scenarios is to consider a lightweight Java workflow engine, which can be easily included into your product".
    So, Bern talks about starting a lightweight Java workflow engine,
    and a JavaSpace implementation do provide such a lightweight Java workflow engine!

    Back to top
  • Flow Strategy first

    by William Martinez,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Ok.
    Before thinking in workflow engines, I would say to think first in strategy.
    See this post about it. Maybe the flow can be implemented differently.


    William Martinez Pomares.
    Architect's Thoughts

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Bernd Ruecker,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hi Dominique.

    JavaSpaces is definitly an interessting technique, I like it very much. But more for the Sevicecall aspect of processes, not the sequence flow control. Because there you still have the responsibility to code the workflow code yourself.

    And Workflow/BPM is not only about sequence flow control and state machines, but about reporting, monitoring, audit logging, administration and so on... This you get for free if you start with an existing workflow engine (okay, not wirh all of them everything ;-)).

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Dominique De Vito,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hi Bernd,



    Of course, I do agree with you: JavaSpaces does not provide all components of a workflow.



    My idea, here, is the following: adopting JavaSpaces, as a JavaEE spec, would be a pretty good step forward for promoting the use of already existing workflow components. So, developers could be more keen to adopt them, to leverage them, instead of hand-coding custom workflow and starting coding from scratch.
    Some people are not using workflow engines because they think adopting them is quite a too big step in one move. Introducing JavaSpace inside JavaEE could change that idea, and how developers see workflow engines.



    And while basic JavaSpaces may provide only (workflow) context storage, and context locking mechanism, I do expect that more advanced JavaSpaces could provide too reporting, monitoring, audit logging, administration and so on... Just like the JavaEE spec details "basic" features, and some JavaEE servers do provide also clustering, failover, reporting, monitoring, audit logging, administration and so on...



    Dominique

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Bernd Ruecker,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Okay, using somethign instead of building everything yourself is most of the time a good idea ;-) But JavaSpaces has some learning curve as well..

    Some people are not using workflow engines because they think adopting them is quite a too big step in one move


    Yeah, I know. Unfortunately...


    I do expect that more advanced JavaSpaces could provide too reporting, monitoring, audit logging, administration and so on...


    But I guess not a business level. Business processes are not only about technical moitoring, but also business requirements, KPIs and stuff like it. This is the target of workflow engines. Your proposal semms a bit technical...

    But anyway :-)

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Dominique De Vito,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Okay, using somethign instead of building everything yourself is most of the time a good idea ;-) But JavaSpaces has some learning curve as well..


    Sure, JavaSpaces is not a magic bullet ;-)


    I do expect that more advanced JavaSpaces could provide too reporting, monitoring, audit logging, administration and so on...


    But I guess not a business level. Business processes are not only about technical moitoring, but also business requirements, KPIs and stuff like it. This is the target of workflow engines. Your proposal semms a bit technical...

    But anyway :-)


    As you say, my proposal seems a bit technical... but, just look around: JavaEE is full of technical specs!
    And the fact that JPA is a technical spec does not stop developers to persist business models!


    Business layers just sit on top of technical ones.




    One can't include whatever he/she wants into JavaEE. For example, a new spec needs to be quite compatible with other existing ones, and the new spect needs to be a technical one (not too high-level, not into the business levels).


    So, I don't think a workflow will be ever included into JavaEE. IMHO JavaSpaces looks like quite the single chance to include inside JavaEE (that is, to promote) what looks like relatively close to a workflow. So, that's why I "push forward" JavaSpace; this being said, I am not against workflows ;-)

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Bernd Ruecker,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    So, I don't think a workflow will be ever included into JavaEE.


    No, sure not. But this is no reasonable target at all! There is a bunch of other specs for this (loke BPMN 2, BPEL, XPDL, ...). Workflow normally has a brider scope than pure Java EE, mayber to talk to inhomogenous systems.

    Java EE workflow is a special case, but e.g. BPMN can be bound to a Java implementation, that will get interessting. Standard and workflow.

    So, that's why I "push forward" JavaSpace; this being said, I am not against workflows ;-)

    Good, so we can stay friends :-)

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Dominique De Vito,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    So, I don't think a workflow will be ever included into JavaEE.


    No, sure not. But this is no reasonable target at all! There is a bunch of other specs for this (loke BPMN 2, BPEL, XPDL, ...). Workflow normally has a brider scope than pure Java EE, mayber to talk to inhomogenous systems.

    Java EE workflow is a special case, but e.g. BPMN can be bound to a Java implementation, that will get interessting. Standard and workflow.


    Well, the point is I am sick of programming through XML; just read "Just Say No to XML" by Allen Holub.




    Java (own) strengths have been forgotten. Here are few examples from here:


    - RMI has been somewhat forgotten, and web services have been much more advertized (with cumbersome programming for little productivity for Java-to-Java communication),


    - JMS has been included into Java EE even if it is not object-oriented, while its object-oriented twin, JavaSpace, has been limited to Jini world,


    - a RDBMS has been included into JDK, while no object database has been promoted in a similar way: it's unfortunate the single storage promoted, here, by an object-oriented programming language is only a relational storage mechanism.


    - and XML has been used all over the place, has been pushed forward a bit too much and even promoted inappropriately as a programming language.


    - etc.




    I want to choose Java as a programming language, and next, not to have to choose XML to do a (simple) loop calling a WS. While you need to talk to inhomogenous systems, it's not a reason for XML programming. And if you like dynamic workflow deployment, you could still avoid XML programming: just use OSGi and/or a Java-like dynamic language.




    That's another reason I like JavaSpace (over some workflows with XML abuse).

    Back to top
  • Re: workflow engine

    by Marlon Dumas,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Fully agree with Winston... In telcos, which generally have a strong engineering culture, there is a tendency to either reinvent the wheel (build a workflow engine from scratch) or adopt an overly complex solution (e.g. a heavy BPEL engine with SOAP all over the place). One notable exception to this rule is first:telecom in the UK which several years ago decided to adopt YAWL instead of building their own workflow engine. YAWL is a rather small, fully Java-based workflow engine. Five years later they're still happily using it. The argument that it takes too much time/effort to evaluate and adopt an open-source workflow engine is very short-sighted when you think that such decisions last for at least 5-10 years. I can admit though that it is difficult to justify such decisions for a single project - but when you do it on a cross-project basis and with a long-term view, it can really pay off.

    Back to top
  • Re: workflow engine

    by Bernd Ruecker,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    The argument that it takes too much time/effort to evaluate and adopt an open-source workflow engine is very short-sighted when you think that such decisions last for at least 5-10 years.


    Exactly! I agree completly.

    Back to top
  • Re: why not using a JavaSpace as a lightweight Java workflow engine !?

    by Bernd Ruecker,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    Well, the point is I am sick of programming through XML; just read "Just Say No to XML" by Allen Holub.


    Depends what problesm you try to solve. Java is good in its place, XML good in another...

    Back to top
  • Imixs JEE Workflow

    by Ralph Soika,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    See the Imixs Workflow Project. Imixs Workflow is an open source BPM technology to build modern business applications.

    Back to top
  • Re: Imixs JEE Workflow

    by Ralph Soika,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    See project site from the Open Source Project Imixs Workflow, based on Java Enterprise Edition
    www.imixs.org

    Back to top

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT