BT

Choosing the Right ESB for Your Integration Needs

Posted by Kai Wähner on Apr 02, 2013 |

Different applications within companies and between different companies need to communicate with each other. The Enterprise Service Bus (ESB) has been established as a tool to support application integration. But what is an ESB? When is it better to use an integration suite? And which product is best suited for the next project? This article explains why there is no silver bullet and why an ESB can also be the wrong choice. Selecting the right product is essential for project success.

Definition of the Term "Enterprise Service Bus"

Numerous products from different vendors include the name "Enterprise Service Bus". Unfortunately, there is no standard definition of this term. The products therefore offer many different features. The term ESB should always be clearly defined before it is used. In the following, an ESB is defined as a software product which assists the developer in application integration and therefore provides the necessary infrastructure to implement routing, translation, and other integration facilities. On the integration complexity path, an ESB usually falls between a framework and a suite as an alternative for application integration, as shown in the following picture:
 
Figure 1: Alternatives for application integration
After the term ESB has been defined, the next section explains when an ESB should be considered, and when an integration framework or an integration suite is a better choice.

Integration Framework

A framework helps implementing Enterprise Integration Patterns (EIP, http://www.eaipatterns.com), such as Splitter or Content Based Router, in order to integrate applications in a standardized way. Using standard APIs to integrate products noticeably lowers the implementation effort and the source code is easier to understand for other developers. A framework is well suited to integrate different applications with different protocols and technologies, and concepts such as endpoints, producer, consumer and EIPs are used to create the integration logic. Even implicitly supported test automation uses the same concepts.
A framework consists of a set of ordinary libraries and is therefore compatible with any development environment, even a classic text editor.
Known examples of frameworks are Apache Camel and Spring Integration in the Java environment and NServiceBus for .NET.
With frameworks the development team is more or less solely responsible for the project's success. Commercial support is generally not available. Tooling is also only partly available, and not necessarily suitable for "mission-critical" deployments. The rest of this article is therefore devoted to ESB and corresponding suites, which are often a better choice than a framework.

Enterprise Service Bus

Like a framework, an ESB is used to integrate applications. An ESB is based on a framework implicitly. However, it is a much more powerful product. Contrary to a framework, an ESB offers strong tool support for deployment, administration and monitoring at runtime, besides basic functionalities for application integration. In addition, graphical editors are used for the implementation of various integration scenarios. The integration logic can be modeled with "drag and drop", the corresponding source code being generated automatically. Commercial support completes the package of an ESB.
The great advantage of ESBs over the use of pure frameworks is therefore the better tooling, which reduces the cost and complexity significantly. Integration problems are solved at a higher abstraction level.

Integration Suite

A suite offers all features of an ESB. In addition, many other functions such as Business Process Management (BPM), Business Activity Monitoring, Master Data Management, or a Repository are included. If some of these additional features are required in addition to pure integration, then the use of a suite is advisable. The entire integration can be realized with a single software stack.
The differences between a framework, an ESB and a suite are hopefully now clarified. Next will be explained how to select the right ESB or suite.

Comparison Criteria

Notice: we will not provide a matrix that compares all the available products with respect to various criteria. ??From the perspective of the author, it is hardly possible to create a good and useful matrix because the products offer far too many (often different) functionalities and concepts. Besides, the feature list also changes virtually every day in the IT world.
Therefore, it is advisable to pre-define your own needs, and then to evaluate which products are best suited. Proprietary solutions are often very similar, and also the most used open source competitors offer similar characteristics. Therefore, it makes sense to think of at the beginning, whether a proprietary or an open source solution is the better choice.
In order to make this decision, you should use the following criteria:

  • Usability: How complicated is the installation? How many tools are needed? Is the development environment intuitive?
  • Maintainability: How do you administer the product? Is there a GUI for monitoring services?
  • Community: Are there active public forums or mailing lists? Are numerous articles, tutorials, articles, and videos available? Is the product supported by several companies?
  • Enterprise Support: What support options are offered ("business hours", "24/7" hotline vs. Email vs. on-site support, etc.)? Can the required service level agreements be guaranteed? Is support offered in your preferred language?
  • Functionality: Are all the required functionalities offered?
  • Flexibility: Can you customize functionalities of the product to fit my needs?
  • Expandability: Is it possible to expand the product? Is the product and its interfaces based on standards?
  • Connectors: Are adapters for all required technologies available? Are there adapters for B2B products such as SAP or Salesforce? How easily can I build your own adapter?
  • Cost: What is the full cost (total cost of ownership) of the product - including maintenance, all required ancillary products, connectors, etc.)?
  • Licensing: What licensing or subscription model is used? What happens when requirements change (more computers, more CPUs, switching to virtual machines, etc.)? Are upgrades for free? Are downgrades possible, too? Are the costs "foreseeable" at all, is the price list even understandable?


Table 1 compares the advantages and disadvantages of proprietary vs. open source ESBs and suites (green = good, yellow = medium, red = bad). The conclusion regarding the differences is the following: Proprietary solutions generally offer more features and "powerful" support. However, the question remains, "Are these really needed?" Remember that efforts, complexity, and costs are correspondingly higher. Open Source products score with easier usability, greater flexibility, easier extensibility and lower costs.

Criteria

Proprietary

Open Source

Ease of use

Very complex installation (consultants needed !?), „tool hell“

One Click Installer (also for Mac in some cases), start using after minutes, unified platform

Maintainability and Monitoring

Powerful tooling (e.g. for administration and monitoring), Analysis of source code not necessary, refactoring via GUI

Less powerful tooling (e.g. for administration and monitoring, integration of further products from other vendors required sometimes), Analysis of source code not necessary, refactoring via GUI

Community

Buy support, forums (but no real community which helps)

Based on open source projects, plus own community

Enterprise Support

24/7 enterprise support, SLAs as you wish, deployments with thousands of servers

24/7 enterprise support, less guarantees than proprietary support, check for local consulting and support

Functionality

Integration features + many more (BAM, CEP, EDA, etc., etc., etc.)

Integration features + some more

Flexibility

(Make a change request + wait long + pay) OR (pay a lot + get it quickly)

Open source, change what you want

Extensibility

Do it yourself (tough) OR pay

Standards-based, defacto standards

Connectors

Adapters for technologies and business products

Adapters for technologies and business products

Costs

MUCH (and even more)

LESS

Licencing

Complex price list, pay for everything (upgrades, migration to VM, „you-name-it“)

Subscription model, upgrades inclusive, predictive costs, downgrades possible

 Table 1: Advantages and disadvantages of proprietary and open source products

Product Alternatives

After the main differences between proprietary and open source products have been explained, some specific products are introduced in the following. Thus, an overview of the various available alternatives will be given including a small practical insight.
Almost every vendor of proprietary integration products, such as IBM and Oracle, offers a solution for every conceivable function. Regarding open source alternatives, in particular the Unified Platform of Talend and the WSO2 Platform are worth mentioning, because they also offer complete suites. Besides, several alternatives concentrate solely on ESB. Perhaps the most important open source offerings are Mule ESB and Fuse ESB.

Oracle Service Bus / Fusion Middleware

Oracle Service Bus is the current ESB from Oracle. It is a component of Oracle Fusion Middleware (OFM) stack, which according to the definition of this article is an integration suite. Many other products are available in it, for example, the SOA Suite, Coherence, Complex Event Processing, BPEL Process Manager, Enterprise Messaging Service, Service Registry, and many more.
There is probably not much functionality which is not provided by Oracle’s suite. The tools are very powerful and stable. Graphical editors exist for most products. Support is also available for most conceivable service level agreements. If these powerful and SLAs are really needed, you are on the right side with Oracle. This power comes at a price of course. The high complexity of the products should not be underestimated. Besides, you should be aware of high licensing and support costs plus a non-transparent pricing model.
The OFM is based on standards such as Java EE, BPEL, SOAP, or SCA. The products are proprietary and come from multiple acquisitions made by Oracle over time. Therefore different codebases are used, and different products often need different development tools. The sum of the downloads can quickly exceed 20 Gb. The installation is tedious and can occasionally move over several days - even for a simple installation on your laptop. The products are rather heavy. Resource requirements at runtime are very high.
By the way, you could have just replaced "Oracle" with "IBM" and "Fusion Middleware" with "WebSphere". The content of this section would still be almost the same - except to mention that IBM has three different ESBs in the portfolio: the Message Broker, the ESB and the DataPower SOA Appliances. Also, Tibco, Microsoft and SAP play an important role in the market of proprietary ESBs and integration suites.
The conclusion of this section is therefore that proprietary integration suites can offer almost every conceivable function and cover almost all SLAs. However, many of these functionalities or SLAs are not required in most projects. In this case, be sure to also evaluate open source alternatives. The most important ones are described in the next sections.

Mule ESB

Mule ESB is one of the first successful open source ESBs. It has a lot of qualities in common with the other previously mentioned open source ESBs. These include a very simple ("one click") installation and intuitive, Eclipse-based tooling. Usually, open source ESBs are very lightweight and extensible solutions. Apart from the free open source version, a commercial enterprise version is available. This offers additional functionality and support for the product.
For those who still do not know it, it should be mentioned that “open source” does not mean “for free”. Even vendors of open source software have to make money and cannot develop products and offer support free of charge. However, the prices are much more customer friendly and not based on obscure price lists as most proprietary products. Nevertheless, the open source version can be used (even in production) without any licensing costs. Often, however, the open source version simply serves for playing around or doing a proof of concept to later upgrade to the enterprise version with additional features and support.
As the name suggests, Mule ESB is a pure ESB. Important advantages in contrast to frameworks like Apache Camel or Spring Integration are the graphical editors for an efficient implementation of integration scenarios and the available connectors for B2B products such as SAP or Salesforce. However, the functionality of a suite is missing in Mule ESB. For such use cases, the ESB has to be combined with products from other vendors. Negative aspects of Mule ESB are the small community, a restrictive licensing model and limited availability of the source code. Competitors have significant advantages at this.

Fuse ESB

The Fuse ESB is a pure ESB like Mule ESB, without a suite. It is based on de facto standards in the integration environment such as Apache CXF and Apache Camel. As a result, a great community is already secured from the ground up. The development environment is based on Eclipse and very intuitive.

Fuse ESB was part of FuseSource. However, FuseSource was recently acquired by Red Hat and now belongs to the JBoss division. Fuse ESB is contained in the current road map and will continue to be supported. It will be integrated into the JBoss Enterprise SOA Platform - just like the also acquired BPM solution Polymita. There is still a long way towards a unified suite, since the integration of FuseSource and Polymita will still take a few months, and with JBoss ESB, Switchyard and Fuse ESB now three ESB products need to be merged into one. Here, other open source vendors have already achieved better results.

Talend ESB / Unified Platform

Talend ESB is part of the suite of Talend. Talend ESB can be used independently or in combination with other components of Talend’s unified platform. All components are open source and freely available. The enterprise version offers additional features and support. The difference to proprietary products is that all the partial components are based on the same code base and the same tooling is used everywhere. The usage of different areas such as ESB, BPM, ETL, MDM, etc. is done fluently - and is not a separate integration project on its own.
All tooling of the Talend suite is built on Eclipse. The familiar "look and feel" and the intuitive use of Eclipse remain. Talend offers a visual designer for all product parts and uses a "zero-coding" approach. This allows an efficient implementation of integration scenarios. Of course, you can still write and integrate custom logic to your project, e.g. via Java classes (POJOs) or different scripting languages.
Like Fuse ESB, Talend ESB is based on several de facto standards in the integration environment such as Apache Camel, Apache CXF, Apache Karaf and Apache Zookeeper. Besides the available connectors for Apache Camel for technologies such as JMS, HTTP or FTP, many other B2B adapters are available, for example for Alfresco, Jasper, SAP, Salesforce or host systems. All 500+ connectors are included by default. One consequence is that Talend’s IDE has higher hardware requirements than its competitors. You should not install Talend on a too weak laptop. Another disadvantage is missing SOA governance features. This is planned for next releases.

WSO2 ESB / Platform

WSO2 is a relatively unknown vendor. However, WSO2 provides the entire range of components of a suite including Business Process Server, Business Rules Server, Business Activity Monitor or Governance Registry. The entire WSO2 platform can be installed very easily and offers a lightweight, Eclipse-based development studio. Like Talend and FuseSource, WSO2 also puts primarily open source projects such as Apache Synapse (lightweight ESB), Axis (Web Service Implementation) or ODE (Business Process Engine) into its components.
Besides Talend, WSO2 is the only vendor that offers a full suite that is based on a single code base and a single development environment. Therefore, nothing stands in the way for an iterative development process, beginning with a small couple of features, and by adding more functionality later step-by-step. A weakness is the graphical tooling. It supports all components of the platform, but it is not as intuitive to use as the tooling of its competitors.

“Do it yourself” Integration Suite?

A warning to the conclusion: The combination of several frameworks or products to build your own custom integration suite is usually unnecessary expensive and leads to many additional pitfalls. Since several solutions already exist, it is strongly discouraged to create one from various pieces. "Glue code" needs to be written, testing and bug fixing have to be done, and there is no specific contact in case of problems. Vendors usually just refer to the other side, i.e. if you try to combine an ESB with a BPM solution of another vendor, which one do you call when you have a problem? So why should you care about all of these problems if other people have already cared, and an entire stack (also open source) is already available?

Conclusion

There is no silver bullet to solve integration problems. First, a decision must be made whether a framework is sufficient. Be aware that most of the source code must be written by yourself, and tooling and support is scarce. Otherwise, an ESB is the better choice. However, if any additional features of a suite are required at some point later, it would be better to use the ESB of an integration suite from the beginning. This secures sustainability without any problems and additional expense for the combination of multiple products.

If an ESB or integration suite should be used, it must be decided whether a proprietary or open source product is a better choice. Proprietary solutions provide all possible features and strong support. However, this also leads to higher costs and a perceived higher complexity. Contrary, open source solutions score with lower cost, ease of use and flexibility. Once this issue is settled, a shortlist can be created to evaluate the alternatives in detail. It is strongly recommended to perform a proof of concept before making a final decision. Ensure that your team will implement this prototype (from the first installation to final deployment and monitoring), and not just the consultants of the vendor. Your team will have to install the product in the future alone and implement the integration problems independently of any consultants which may not be available.

About the Author

Kai Wähner is a Principal Consultant at Talend. His focus is in the areas of Java EE, SOA, Cloud Computing, BPM, Big Data, and Enterprise Architecture Management. Please give feedback via email (kontakt@kai-waehner.de), Twitter (@KaiWaehner) or social network (LinkedIn / Xing).

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Fuse ESB has always been open source by James Strachan

Hi Kai

Any chance you could the correct various mistakes & confusion in your article around Fuse.

e.g. "Fuse ESB was a paid, proprietary product while under the care of FuseSource". Thats never been true. Fuse ESB has always been open source and completely Apache Licensed; not sure where you got this from. Could you please update the article?

It also seems a bit confused what an ESB is. Polymeta is a user focussed BPM suite; so its nothing to do with the ESB so there's no unification between it and Fuse required? As of JBoss Fuse 6.0 which is being released as I speak, there is really one unified ESB at JBoss; Fuse ESB. Switchyard is a service composition framework which works with Camel / Drools / jBPM.

Re: Fuse ESB has always been open source by Kai Wähner

James,

Sorry, the part about "proprietary Fuse ESB" was a mistake. Article will be changed, of course! I already send an email with an updated section to the editors. I do not know why this is in my head. However, you are right. Probably, I mixed it up with Fuse IDE ?!

I have read about JBoss Fuse (new brand for Fuse ESB). I looked at this some days ago, it was not available already. Great news that it will be released these days. Maybe you can post a link here to the press release as soon as it is released.

Regarding unification of ESB and BPM, I disagree with you a little bit. Of course, an unification is not required. However, that is the advantage of unified suites. Developers use one tooling to implement ESB, BPM, and other components. So, an unification is not required, but a huge benefit (if you need more than just an ESB).

Re: Fuse ESB has always been open source by James Strachan

Thanks Kai. Here's the JBoss Fuse site, I'll post a link to the announcement when it comes out.

Just to be completely clear, since the acquisition by Red Hat, JBoss Fuse (Fuse ESB), Fuse IDE and all the management tooling is completely open source. Or to say that even more clearly, there is no code which is not open source :).

Incidentally for those who didn't know, our long term open source management console for Apache Camel, Apache ActiveMQ, Apache Karaf, Fuse Fabric, middleware & integration is called hawtio

I agree folks want a way of using the plethora of technology out there together & to have them integrated & managed in a consistent way. This is one of the reasons Red Hat's open source portfolio is so compelling; Red Hat are the only vendor who has a completely open source distribution and support for the entire stack of middleware & integration technologies: from RHEL, OpenJDK, virtualisation, application servers (EWS, Fuse, EAP), middleware, ESB/integration (JBoss Fuse), SOA, governance/BAM, BPM and rules/BPMNS, distributed caching, virtual storage & networking - together with cloud, PaaS, IaaS and soon iPaaS.

Enterprise Support by Claus Ibsen

Hi Kai

I wonder a bit why you have categorized enterprise support for Open Source as yellow?

Its my understanding that you can obtain support on par with the big closed source vendors, on the open source side as well.
My employeer, Red Hat, have customers who subscribers this high level of support from us.
For example our federal and government customers of ours demand that, with high level of SLA as well.

Re: Enterprise Support by Mark Little

Just to reiterate what Claus has stated, Red Hat offers 24x7 support to our customers on our open source products. That's part of our value proposition and why we became the first open source company to break the $1 billion barrier last year (www.linuxtoday.com/it_management/red-hat-grows-...)

The article really needs to be updated and stick to the facts. Thanks.

Re: Enterprise Support by Kai Wähner

Hi guys,

thanks for your feedback.

With "yellow" for support, I do NOT mean that an open source vendor does not offer good enterprise support! Claus' example (federal and government customers with high level of SLA) is also true for my company (Talend). So the question is not if these companies do not offer good enterprise support. They do offer good enterprise support, definitely.

So no question: Most open source vendors also offer 24/7 support, etc. (as it is stated in the table). However, usually, open source vendors cannot offer same "powerful" support as several proprietary vendors such as IBM or Oracle. As described in the table, this includes availability of consultants, support in language of customer, some guaranteed SLAs.

In my experience, vendors such as IBM or Oracle can offer more here - of course, it has its price. For example, I have seen many open source vendors which cannot offer German support.

Unfortunately, a "generic table" always has some exceptions, however IMO the table is true for most available products, therefore it is a good starting point for doing evaluations. In the end, a decision maker has to evaluate his short list in more details to understand the differences between alternative products.

Re: Enterprise Support by Mark Little

Hi Kai,

I'm sure our customers and consultants in China, Japan, Brazil, Spain, Germany, France, Sweden, India and a host of other non-English speaking countries would like me to remind you that many of them have moved from the likes of IBM and Oracle for many of your so-called "powerful" features. And I'm not suggesting that Red Hat is alone in this regard: there are other open source companies that do the same. However, I do think it is important that your article is as accurate as possible and it is clear by now that a two column table is not able to convey the facts.

Re: Enterprise Support by Kai Wähner

Well, I am not really sure, if Red Hat can give support in all these languages for e.g. JBoss Fuse :-) However, I agree with you that this article cannot be accurate enough to describe all facts. You have to write a book for this. Though, I still think that this article is perfect to begin thinking about integration solutions. You can get an overview about different kinds of solutions and aspects which you have to look at. With this article, you can begin to create and evaluate your short list. Then you will find out, if for example this or that vendor has support in this or that language. Afterwards, you can make a good decision.

That's also the feedback I got from many readers...

ServiceMix? by abhishek manocha

Where does ServiceMix sits among those 3 definitions then?

I have started to see this term (ESB) being used for almost all middleware by Ian Fleming

Recently I have noticed a surge in the term ESB from our prospects who describe their systems architecture.
I think it has replaced SOA (which replaced EDI) as the generic placeholder for an api messaging architecture.

That said, am I right in thinking that most ESB messages can be tested via SOAPUI or is there a better tool for this.

Re: ServiceMix? by Kai Wähner

The plain Apache project ServiceMix is really tough to categorize :-)

Besides integration features, it also includes messaging and BPM engines. However, Apache ServiceMix misses two main advantages of ESBs / integration suites: tooling and commercial support. Instead you have to do a lot of coding by yourself. So for me, the definiton of a framework fits best - even if that's not really true. ServiceMix is somewhere in the middle of these three alternatives as it is Apache Camel (integration framework) and some more stuff around it...

Fuse ESB was the enterprise version of ServiceMix offered by FuseSource (which added tooling + commercial support). I am not sure if JBoss Fuse will also continue to use ServiceMix, or maybe just Camel, ActiveMQ, etc. Probably, one of the JBoss guys can give some more details here?

Re: I have started to see this term (ESB) being used for almost all middlew by Kai Wähner

Well, SoapUI is an awesome tool for testing. It is a perfect match if you "just" want to test SOAP / REST web services, JMS, etc. I use SoapUI especially when I just want to do some rapid prototyping and testing. I don't know a better tool for this.

Depending on your product choice, there might be better alternatives for your "real" integration tests.

Integration frameworks such as Apache Camel offer awesome testing capabilities. You use the same concepts as in your integration code. The testing code is always almost the same - no matter which technologies you integrate. I prefer this solution if developer and tester are the same person.

Many ESB vendors also offer good testing tools which are already integrated into their solution. For example, in Talend ESB, there are components for creating service consumers / mocks / dummy data easily without coding. This tooling is integrated directly into your IDE. You create tests the same way as you create your integration logic. Nevertheless, Talend has integrated SoapUI into its IDE, too. You can choose depending on your use case.

Evaluation Framework by chris haddad

Kai, you provide good evaluation criteria, a thoughtful review of product advantages, and valuable perspective about the continuum between lightweight service consumption and more complex integration scenarios. Your readers may desire to review an alternative evaluation framework posted at blog.cobia.net/cobiacomm/2012/04/18/how-to-pick...

print friendly by will mason

I would love it if infoQ could do printer friendly rendering; I'm going on a trip tomorrow and would have liked a readable version as printed out, or even a PDF copy for my smart phone. Please add to the wish list.

avoid mule esb by Abdurrahman Sahin

As a recent mule esb user,
I want to warn people not to waste their time with that incomplete defectful product. I d suggest having a look at Spring integration framework for integration projects.

Re: avoid mule esb by David Dossot

Would you agree to share this list of defects with me over email (david@dossot.net)? Thanks for your feedback.

WSO2 Enterprice service Bus by Rashod Chamikara

at WSO2 ESB a alternative solution for mule you can find fallowing connection features

Transports: HTTP, HTTPS, POP, IMAP, SMTP, JMS, AMQP, FIX, TCP, UDP, FTPS, SFTP, CIFS, MLLP, SMS

Formats & protocols: JSON, XML, SOAP 1.1, SOAP 1.2, WS-*, HTML, EDI, HL7, OAGIS, Hessian, Text, JPEG, MP4, All binary formats, CORBA/IIOP

Adapters to COTS systems: SAP BAPI & IDoc, PeopleSoft, MS Navision, IBM WebSphere MQ, Oracle AQ, MSMQ

Adapters to cloud services: Salesforce, Paypal, LinkedIn, Twitter, JIRA

ESB Product Architecture and Strategy as an Evaluation Criterion by Mariusz Brylant

Having been extensively working as an architect, designer and developer with different integration/ESB products, including IBM MQSI, IBM WebSphere, MS Biztalk, SAP PI, Apache ServiceMix, Zato and Mule , IMHO one should consider the architecture of a product, its history and evolution strategy.

This is maybe more general criterion and applies to not just ESBs and boils down to whether the product have been uniformly architected from the ground up or have rather been assembled from a collection of components which have NOT originally been designed to work together as a coherent whole.

The latter tends to have greater TCO often due to less then seamless experience off working across different tools of a package.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

18 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT