BPM Is Not Software Engineering

I thought I agreed after reading the first few paragraphs. I can perfectly accept the definition of BPM that turns it into a modeling discipline targeted at communicating what's need for business people to decide on how to improve processes, or help them describe the domain knowledge to software engineers. But the author lost me when he wrote:

A business person draws a diagram, that is the diagram that is executed.

Stefan,
Executable does not necessarily means automated. There are thousands of business processes that are executed, for example, using email.

If it's executable, it's by definition so formal that it is a program. If it's a program, the person creating it is a programmer.

You have me confused – I'm not sure whether we agree or disagree. If you define "executable" as "will be done somehow", the problem of course disappears and we agree.

If it's executable, it's by definition so formal that it is a program. If it's a program, the person creating it is a programmer.

I think, that this exact train of thought is the root cause Of today's BPM problems. A diagram, drawn by business person can be executed only in the most simplistic cases. In majority of cases, such diagram has to be precise enough to outline intent, but typically requires software engineer to do the actual implementation. Even with the modern BPM suites, implementation of the business process is complex enough to require professional developers.

From my own observations, giving a business analyst a BPM tool - and not involving a Software Engineer in the design of the process flows - may result in business process definitions that don't include due consideration for edge-cases, normal error conditions, and exception paths.

John,

The required skills are pretty much HTML/Javascript, SQL, and a bit of XML and web services - Very procedural, mostly "scripting" - Programming... Yes - Software Engineering... Not so much.

The BPM tools are often marketed as the ultimate solution for the business to take control of their IT requirements. They don't need software engineers to make changes. It can all be done using these tools. Yeah, right.
What the business people fail to understand is that they can not implement a process unless they get the DBA or software engineer to change or implement features needed for the process to be tied together. For example, in order to approve an order the information needs to flow from the order system and back. I know I am thinking about this as a software engineer and in terms of data. But what is a process without data?

I don't quite get the bottom line of Keith's article. If BPM is not SW-Engineering, then the inevitable conclusion must be that *NO* BPM model must be (machine-)executable, not even in parts. Once it is, there is Software and designing Software follows some principles, which business people are not familiar with, no matter what the used language looks like (textual or graphical).

Just because some language looks simple in the first place, does not mean that it cannot screw up things.

I am sure that Keith knows all this, that's why I am a bit lost.

I think, business people should never be exposed to general-purpose programming APIs or languages. Specifically designed DSLs with a narrow scope are fine. UML is not (if used as generator input). BPEL is not (making it graphical doesn't change a thing). Scripting languages? Inappropriate.
A BPM modelling tool is general-purpose, since it is not specific to any domain. A domain-specific process modelling tool would be a DSL with a graphical interface. Fair enough, but that's not what this discussion is all about, is it?

Agreed. Even if they did, the diagram would quickly lose its value as a communication medium, as the edge-cases/exceptions etc create too much clutter.

The bottom line is that if people that a) participate in Business Process Standard working groups, b0 build BPM products, or c) claim they "automate" processes could open their mind just a tiny bit behind thinking that every- and any-thing can be reduced to a few "programming" language statements we could probably be in a very different situation today.

Boris, replying to Stefan, gives us some clues about the answer:
>> Executable does not necessarily means automated. There are thousands of business processes that
>> are executed, for example, using email.

Indeed, a "business process" is far better represented by entities that collaborate to achieve a single goal. Unfortunately, our "programming" models have no idea what a "collaboration" is and how "state" is managed across the collaborating agents (human or software). Scripting a couple of steps here and there is not going to be able to represent what is really going on in a "business process".

Sadly, as we were getting ready to layout the foundation on which BPM could have happened (bidirectional interfaces, assemblies, orchestration languages (to implement the behavior of the collaborating agents)...), it seems that our industry has decided to go back to a synchronous, uni-directional, CRUD oriented, contractless middleware layer, deeply and totally incompatible with BPM.

So it is not exactly a "developer" vs "business" mindset problem, it is really a formalism problem where traditional programming models are simply unable to create an execution environment compatible with BPM (such as the one Boris is talking about). I am not sure Keith has expressed it that way, but maybe just for once, we could ask the question of whether BPM can be supported by traditional programming models. If not, which ones?

I follow you in that a a better, higher-level programming language (or programming model, whether graphical or textual) might be much better suited for BPM purposes than any of the available GPLs. But even the perfect DSL is still a formal language. Even if process flows/collaborations/orchestrations/whatever-you-like-as-appropriate are its main concepts, the formulation of a business process in a form that is executable is still a program. And I remain convinced that for building good programs, a sound knowledge of software engineering principles is a prerequisite. Do you disagree?

Stefan:

could it be possible to agree that there are 3 major formalisms for describing "execution":
- Turing
- Petri
- Milner

Would you think that "a sound knowledge of software engineering principles is a prerequisite." would mean the same thing in each case? How can the principles of a "connected system" be the same as the ones of a signal processing application? I would argue that 99% of developers have a decent understanding of Turing-based approaches to programming, about 0.9 % kind of get Petri-based programming, and about 0.1% understand Milner (could even be lower).

When given a choice, a developer will always implement a unit of work as a series synchronous calls with a central thread of execution. Unfortunately, this implementation model does not work at all for BPM (we tried for 10 years, lots of smart people tried during that time, and we got nowhere).

IMHO what Keith means is that business people can express fairly fuzzy specifications for a business process. Humans that perform the business process will somehow fill in the blanks and adapt to lots of situations while achieving the goal(s) of the business process. Developers, with the current programming models, are not be able to achieve this kind of behavior. What they produce is rigid and cannot adapt easily to exceptions, nor is it easy to express all possible exception paths.

Maybe, just maybe, "developers" could decide once and for all that a (turing based) "centralized" execution engine is the fundamental reason for this roadblock, and maybe just maybe, they could decide to look a bit further. They could introduce, for instance, modern concepts such as "collaborations" between software agents. They could also reflect on the relationship between "states", activities/actions and business processes. Unfortunately a recent survey of would-be thought leaders in the field of BPM/SOA can't even correctly distinguish between a state and an action. I don't know how we will ever make any progress in these conditions.

The point I agree with is this one:

Humans that perform the business process will somehow fill in the blanks and adapt to lots of situations while achieving the goal(s) of the business process.

Don't be frightened Stefan :-)

In an attempt to clarify my thoughts I posted a more reasoned response on my java.net blog.

Jean-Jacques,

you are taking the scientific route and look much further ahead than I did. I regarded the platforms of today. The attempts to process automation which I have seen so far all have the deficiencies you mentioned. However, those products are purchased by companies in expectation to automate their processes further. As a consultant, I need to adapt to that and give the right usage recommendations for those (maybe crappy) products.

This scenario makes BPM a lot like SW-engineering. I don't get the budget to research and establish a new programming model. I'd love to go another route, really, but the industry follows the heavyweights and if they don't get it right, all I can do is fire-fighting or quit this hopeless subject and look for something else (my current tendency, BTW).

Stefan:

>> The programming model doesn't change this

the programming model that our industry has converged towards is synchronous and assume that the fundamental unit of work is best described by an isolated request-response pattern (~single threaded). It it is true that every computing problem can be reified behind this simple concept, who would believe that a long running, graph-oriented problem could be simply decomposed in such fashion and serendipitously achieve round-tripping and perfect alignment with the way business users would think about the problem?

Could we just get real for one second? This is not about teaching everyone to script a couple of things, this is really about developers and architects understanding that we have reached the limits of the synchronous single-request programming model. The day people will understand that problems are a lot easier to solve in other programming formalism we will have made a big step forward. It is like trying to solve some problems that are best addressed by spherical geometry while trying to use euclidean geometry.

I believe I see now where you are heading. Does your envisioned architecture consist of autonomous entities, which communicate asynchronously with a kind of rendezvous mechanism (sync points) or mailbox-like message exchange?

It still appears abstract to me. Is your research work or that framework you've mentioned once related to this subject?

Hermann:

the problem is with the vendors and industry thought leaders. I have been working in the BPM field for over 10 years now, building products, defining standards, using other vendors' product. It is of course not the role of an end-user to go at this level.

Most people in the vendors camp are looking to recycle whatever technology or formalism they can without looking any further. In the BPM space for instance there was the "pi-calculus" phase. The concepts of Milner theory are very important to understand for BPM. However, Milner's theory itself is totally unusable for BPM. Yet, the BPM camp had to fight at least 5 years to make these people understand that this was a dead end. In the WS-CDL standard I had to create a UML diagram of Pi-Calculus to help people understand that the semantics were incomplete. (Milner never had BPM in mind when he worked on Pi-calculus). So what happened? Some vendors claim that Orchestration languages were sufficient to "execute" business processes. They are only necessary, but not enough...

Nowadays, the REST camp is making us return to a synchronous, CRUD oriented programming model. They are wiping out other critical elements of BPM technologies that SOA had put in place: bi-directional interfaces, assemblies, orchestration languages, forward versioning strategies... On the other hand, the BPMN camp is completely ignoring the concepts of "Resource Orientation" (not REST). Worse, some people are now thinking to use BPMN as a scripting engine for REST.

Those are just a few examples of the insanity that has surrounded the field of BPM. BPM will only take off the day we will have aligned Service Orientation, Resource Orientation and Event Orientation as part of a process-centric programming model. Until then, all efforts are pretty much wasted on all sides (BPMN, BPEL, REST...).

>> quit this hopeless subject and look for something else (my current tendency, BTW).
I am with you, this is for now hopeless. Everybody is trying to push his little technology and control its little territory while the end user looses.

I think with have to make a clear distinction between the problem domain and the solution domain. In the problem domain you model the organization. This organization can be supported by software. A model of this software is a model of the solution domain.

In for example Model-Driven SOA we try to define a methodology/process to transform the problem domain model into a solution domain model. I think it is not possible to automate this process because the models take a very different perspective. Problem domain vs. solution domain, user perspective vs. system perspective, etc.

I have some experience with Business Process Modellers who are not programmers. And I can tell you they make a good job in understanding the requirements and even in documenting a possible process in diagrams, but the resulting diagrams are not executable for logical errors or complexity reasons.

It takes a programmer to understand reuse/interfaces/modules as well as error handling or simple loop/branching conditions and invariants.

You can state that BPM is not programming, but you cannot expect the BPM artifacts to be executable in that case. (Let alone field level programming for transformations).

Greetings
Bernd

It is indeed an interesting topic. Therefore, a workshop about the relationship between "Requirements Engineering" and BPM will take place during this year's German software engineering conference. Paper submission is already closed, but you can also participate in the workshop without presenting a paper. I also blogged more details about this workshop some time ago.

It takes a programmer to understand reuse/interfaces/modules as well as error handling or simple loop/branching conditions and invariants.

You can state that BPM is not programming, but you cannot expect the BPM artifacts to be executable in that case.

You can state that BPM is not programming, but you cannot expect the BPM artifacts to be automated in that case.

That is a perfect way to sum it up