ESB-Oriented Architectures considered harmful

Posted by Jean-Jacques Dubray on Aug 28, 2007 10:42 PM

Community

SOA

Topics

ESB,

SOA Platforms,

Messaging,

Enterprise Architecture

Bobby Woolf, co-author of Enterprise Application Integration Patterns, and WebSphere SOA and J2EE Consultant at IBM, wrote an article that questions the legitimacy of an ESB as the foundation of a Service Oriented Architecture (see note below*).

Clients often want to build only an ESB because that involves a technology challenge without the need for messy business requirements. Building just an ESB becomes an IT field of dreams, where IT builds an ESB and then hopes some SOA will come along and use it. Such an ESB-oriented architecture loses the benefits of SOA. It does not create business value. In fact, it incurs cost without reaping immediate benefit.

Bobby's article is very entertaining. The debate is serious though and has been on going ever since Dave Chappell coined the catchy term.  In reference to the contract-first debate, deploying an ESB is like starting your SOA with "Connectivity-First". Bobby argues the ESB-oriented architecture approach

...is inherently flawed in that it builds connectivity no one might ever want to use... Always implement things when you actually need them, never when you just foresee that you need them.

The OASIS SOA Reference Model does not speak about "connectivity" per se, but introduces the notion of a communication infrastructure:

The primary task of any communication infrastructure is to facilitate the exchange of information and the exchange of intent...Especially in the case where the exchanges are across ownership boundaries, a critical issue is the interpretation of the data. This interpretation MUST be consistent between the participants in the service interaction.

Conventional SOA Reference Architectures, including the one of IBM, always show an ESB in a prominent location. Dave Chappell explains an ESB is in essence a service container with a dedicated communication infrastructure to connect services that share the same container.

A service container is the physical manifestation of the abstract endpoint, and provides the implementation of the service interface. A service container is a remote process that can host software components. In that respect, it has some similarities to an application server container, but with the specific goal of hosting integration services.

Beyond, Bobby's great humor, it is important not to loose track of his arguments. Bobby expressed he disagrees with Joe McKendrick or Dave Linthicum that interpreted that ESB's are useless. I have, myself, written many years ago a series of posts on the topic "Jump off the bus, take a cab" where I questioned the need for a common communication mechanism. Yet, service containers combined with a dedicated communication infrastructure (as Dave Chappell's describes them) are extremely useful, because, Ron Ten-Hove, JBI specification Lead, explains:

• Service containers [are] used to adapt a wide variety of IT assets to the ESB

and an ESB:

• has a reliable messaging system, used to allow the service containers to interact.
• provides message transformation services
• provides message routing services
• provides security features to control access to services
• is centrally administered, despite being a distributed system.
• allows incremental changes to services without requiring shutdown or other disturbance to system availability.

These capabilities (and many more) are essential to a large class of service implementations. D. Sprott, from the CBDIForum, lists a series of patterns that would be difficult to implement without an ESB, for instance using the ESB's Routing Mechanisms to implement a Service Versioning strategy.

Of course, the communication infrastructure starts loosing a bit of its appeal with the completion of the WS-* stack (WS-TX is now available and reliable messaging is near completion**), but efficient service containers will remain key to a successful enterprise class Service Oriented Architecture. I would not be surprised if vendors will soon talk more about their "service containers" rather than their "bus". Not suprisingly, you may nest service containers to combine capabilities.

Bobby's article expresses, with humor, the frustration of consultants facing some IT organization which has little or no knowledge about SOA and is pressed to show progress within unreasonable timelines in any way shape or form. Of course, in the end, the consultants get blamed for delivering something that in itself has no business value. I would argue that Bobby's is giving us a helpful warning that I could summarize as "ESBs are considered harmful if used without vision".  That's probably true of any technology.

* Bobby added some clarification to his article: "ESBs good; ESB-only projects bad. Orient the architecture around the services, not the bus. Is that clear enough?! :-)"

** as pascal pointed out, the stack is complete as of June 2007 with the release of WS Reliable Messaging as an OASIS standard

1. Back to top

   harmful if used without vision

   Aug 29, 2007 4:36 AM by Carlos Rodriguez

   I'm not sure if "vision" is the word. But ESB is like a technical solution for good performance and useful utilities in SOA systems, although you can build a good SOA systems without ESB technology, you can see the Microsoft SOA vision (i am a java-based developer :) ). The correct way, I think that bobby went to say this, is to design a SOA solution and consider the ESB technology (having its features in mind at design-time).

   Reply

2. Back to top

   -

   Aug 29, 2007 8:48 AM by Pascal Schumacher

   "Of course, the communication infrastructure starts loosing a bit of its appeal with the completion of the WS-* stack (WS-TX is now available and reliable messaging is near completion)" I thought WS reliable messaging was already released?

   Reply

3. Back to top

   Re: -

   Aug 29, 2007 9:13 AM by Jean-Jacques Dubray

   I stand corrected, yes last June.

   Reply

4. Back to top

   Sound Advice

   Aug 29, 2007 12:30 PM by Bruce Snyder

   The advice provided by Bobby is very sound and applicable to any project: * Don't build it until you need it * Build what creates business value These two items should be the premise for any sizable project, regardless of whether it's an SOA project or not. The field-of-dreams analogy is never a good idea, especially for business critical projects. But the need to use an ESB to do protocol-to-protocol integration is, IMO, actually a good idea because it prevents you from building silos and accidental architecture. Any technology can be considered a golden hammer. Look at JavaEE; it's been used as a solution to everything. Using the right tool for the right job is imperative and ESBs are much more suited to integration work than green-field application development. The most important point in the article, however self-serving it may be for Bobby, is his presentation of the EIP book and the patterns it defines. Those patterns are far and away the best out there for the typical problems addressed by ESBs.

   Reply

5. Back to top

   It is about the alignment between Business and IT.

   Aug 30, 2007 8:23 AM by Kiet Tran

   In my opinion, one can implement SOA framework without ESB; however, you may end up doing a lot of grunt work. If ESB is to be called service container to convey the message, I am all for that. Remember, SOA is a framework, not technology. It is the classic thinking in that some IT organizations continue to work in isolation. I agree wholeheartedly with Bobby Woolf in his calls for careful thinking in any effort to introduce ESB or service container into the enterprise integration picture. SOA is about the alignment between Business and IT. So, Woolf gave three principles for introducing the ESB as part of the enterprise approach to SOA. I thought they made sense. If an organization does not yet have a clear path to SOA, ESB is of little use.

   Reply

6. Back to top

   Not to go with the Hype

   Sep 24, 2007 9:49 PM by Pravesh Godiyal

   Follow what is best suited for you Business and not what is latest in the technology. It is an hype going where organisation are forcing them to either implement the technology with out much of Business Alignment or they are over implementing without following the guidelines such as re-usability, performance etc. There are few case I have experienced where ESB was constructed and various Web Services are built but without any re-usability in mind. This has added to the performance and further complementing the solution. The solution is following the loosely coupled model but to the extreme.

   Reply

7. Back to top

   Don't build it until you can envision needing it

   Jun 13, 2008 3:24 PM by Anon Ymous

   Would be a better call out I think. I can't get behing the concept of 'Don't build it until you need it' this infers no forethought and takes Ajile to the extreme IMO. This is how you end up with Frankenstein systems with a bunch modules hanging off it that all do something just a little bit different. For people working 9-5 jobs we want to implement technologies that will grow with the business and the concept of an ESB with a purpose aligns to this philosophy. In a general sense, we are saying, build or want with purpose. And to that point this article addresses a common social conundrum of want without purpose, which many people might point to as the cornerstone of all of societies problems:) But 'Don't build it until you need it' I just cant get behind. In that scenario, take literally, you only buy, which may be great for consulting, but not so good in the real world. Not to mention that when applied to 'Projects' in general, we would probably not have most of our modern technology, from Microwaves to the Internet itself if no one had built anything 'before it was needed'.

   Reply

• Older >