ESB-Oriented Architectures considered harmful
Bobby Woolf, co-author of Enterprise Application Integration Patterns, and WebSphere SOA and J2EE Consultant at IBM, wrote an article that questions the legitimacy of an ESB as the foundation of a Service Oriented Architecture (see note below*).
Clients often want to build only an ESB because that involves a technology challenge without the need for messy business requirements. Building just an ESB becomes an IT field of dreams, where IT builds an ESB and then hopes some SOA will come along and use it. Such an ESB-oriented architecture loses the benefits of SOA. It does not create business value. In fact, it incurs cost without reaping immediate benefit.
Bobby's article is very entertaining. The debate is serious though and has been on going ever since Dave Chappell coined the catchy term. In reference to the contract-first debate, deploying an ESB is like starting your SOA with "Connectivity-First". Bobby argues the ESB-oriented architecture approach
...is inherently flawed in that it builds connectivity no one might ever want to use... Always implement things when you actually need them, never when you just foresee that you need them.
The OASIS SOA Reference Model does not speak about "connectivity" per se, but introduces the notion of a communication infrastructure:
The primary task of any communication infrastructure is to facilitate the exchange of information and the exchange of intent...Especially in the case where the exchanges are across ownership boundaries, a critical issue is the interpretation of the data. This interpretation MUST be consistent between the participants in the service interaction.
Conventional SOA Reference Architectures, including the one of IBM, always show an ESB in a prominent location. Dave Chappell explains an ESB is in essence a service container with a dedicated communication infrastructure to connect services that share the same container.
A service container is the physical manifestation of the abstract endpoint, and provides the implementation of the service interface. A service container is a remote process that can host software components. In that respect, it has some similarities to an application server container, but with the specific goal of hosting integration services.
Beyond, Bobby's great humor, it is important not to loose track of his arguments. Bobby expressed he disagrees with Joe McKendrick or Dave Linthicum that interpreted that ESB's are useless. I have, myself, written many years ago a series of posts on the topic "Jump off the bus, take a cab" where I questioned the need for a common communication mechanism. Yet, service containers combined with a dedicated communication infrastructure (as Dave Chappell's describes them) are extremely useful, because, Ron Ten-Hove, JBI specification Lead, explains:
- Service containers [are] used to adapt a wide variety of IT assets to the ESB
and an ESB:
- has a reliable messaging system, used to allow the service containers to interact.
- provides message transformation services
- provides message routing services
- provides security features to control access to services
- is centrally administered, despite being a distributed system.
- allows incremental changes to services without requiring shutdown or other disturbance to system availability.
These capabilities (and many more) are essential to a large class of service implementations. D. Sprott, from the CBDIForum, lists a series of patterns that would be difficult to implement without an ESB, for instance using the ESB's Routing Mechanisms to implement a Service Versioning strategy.
Of course, the communication infrastructure starts loosing a bit of its appeal with the completion of the WS-* stack (WS-TX is now available and reliable messaging is near completion**), but efficient service containers will remain key to a successful enterprise class Service Oriented Architecture. I would not be surprised if vendors will soon talk more about their "service containers" rather than their "bus". Not suprisingly, you may nest service containers to combine capabilities.
Bobby's article expresses, with humor, the frustration of consultants facing some IT organization which has little or no knowledge about SOA and is pressed to show progress within unreasonable timelines in any way shape or form. Of course, in the end, the consultants get blamed for delivering something that in itself has no business value. I would argue that Bobby's is giving us a helpful warning that I could summarize as "ESBs are considered harmful if used without vision". That's probably true of any technology.
* Bobby added some clarification to his article: "ESBs good; ESB-only projects bad. Orient the architecture around the services, not the bus. Is that clear enough?! :-)"
** as pascal pointed out, the stack is complete as of June 2007 with the release of WS Reliable Messaging as an OASIS standard
harmful if used without vision
But ESB is like a technical solution for good performance and useful utilities in SOA systems, although you can build a good SOA systems without ESB technology, you can see the Microsoft SOA vision (i am a java-based developer :) ).
The correct way, I think that bobby went to say this, is to design a SOA solution and consider the ESB technology (having its features in mind at design-time).
I thought WS reliable messaging was already released?
* Don't build it until you need it
* Build what creates business value
These two items should be the premise for any sizable project, regardless of whether it's an SOA project or not. The field-of-dreams analogy is never a good idea, especially for business critical projects. But the need to use an ESB to do protocol-to-protocol integration is, IMO, actually a good idea because it prevents you from building silos and accidental architecture.
Any technology can be considered a golden hammer. Look at JavaEE; it's been used as a solution to everything. Using the right tool for the right job is imperative and ESBs are much more suited to integration work than green-field application development.
The most important point in the article, however self-serving it may be for Bobby, is his presentation of the EIP book and the patterns it defines. Those patterns are far and away the best out there for the typical problems addressed by ESBs.
It is about the alignment between Business and IT.
SOA is about the alignment between Business and IT. So, Woolf gave three principles for introducing the ESB as part of the enterprise approach to SOA. I thought they made sense. If an organization does not yet have a clear path to SOA, ESB is of little use.
Not to go with the Hype
There are few case I have experienced where ESB was constructed and various Web Services are built but without any re-usability in mind. This has added to the performance and further complementing the solution. The solution is following the loosely coupled model but to the extreme.
Don't build it until you can envision needing it
But 'Don't build it until you need it' I just cant get behind. In that scenario, take literally, you only buy, which may be great for consulting, but not so good in the real world.
Not to mention that when applied to 'Projects' in general, we would probably not have most of our modern technology, from Microwaves to the Internet itself if no one had built anything 'before it was needed'.
Camille Fournier May 21, 2015