So Ross can you tell us a little bit about yourself and what you're involved in?

My name is Ross Mason, I'm the founder of the Mule project, an open source ESB

You do a lot of integration work. What are the typical challenges with integration that companies face?

The challenge of integration is on a number of levels. First of all you're integrating two or more applications that don't usually speak to each other and we're designed to speak to each other. But you're also integrating to an infrastructure that is really fixed, you have little control over it and many of the aspects, such as network infrastructures software infrastructure, even political infrastructure has an impact on it. As an integration architect you need to roll these into a solution that's both technical, operational and functional. Those are the key points.

There's a lot of talking about ESBs. What is an Enterprise Service Bus?

The ESB is quite a fuzzy topic. Some people call it a product, some call it a typology. A lot of people call it an architectural style. I'm in the architectural style camp. We're not doing anything new stuff here, we just decouple enterprise systems by having a common message format to communicate between them and then we have these adaptors that do the mediation between the back-end application and the bus. The advantage of this is that it actually scales well; as soon as you put a messaging system in the middle, which is persistent and transactional, you also get scalability and you can cluster to get more performance and throughput. There has been a recent trend towards this because I'd say most of software development these days is not writing new systems. It's integrating with existing applications within an organisation. We've now realised that it's better not to treat two particular applications as a single project to integrate with, but rather go through an integration solution where there's a bus, where you can then in the future bring in more applications as you need to. The reason you do that is to model business processes or streamline business processes.

From mapping requirements to an architecture what kind of requirements would typically map to services on a bus and why would you do that?

A requirement might be that's traditionally trades may be handled by back-office entering it manually, they enter it manually into an accounting system that has some sort of interface to the general ledger and it also goes to auditing and reporting. There's quite a lot of manual process involved. There's no consistent path for the information but yet is something that can be automated; once it is automated you can improve it, you can add more functionality, you can get better reporting statistics you can get more reliable and up-to-date information. The requirements are to really use your information more efficiently and free up your resources to be working on other things. How that maps to the actual architecture is the bus is that communications channel to get all those systems to talk to each other. That's the basic need.

What would the lines on an interaction diagram be between those services and where does the ESB fit in?

The ESB, the actual bus itself, would sit in the middle and all the applications would sit on that bus, so you'll have the trading system sitting on the bus, that is the incoming event that triggers the sort of event flow process; then you have your accounting system that receives information, embellishes it, sends it back; you may then have a general ledger, reporting and accounting. Each one of those coarse grained systems would sit on the bus. But actually it's interesting because the model actually scales down as well to an application level. Some integration we've done was with product companies who buy other products to improve their stack, but these products that they want to include in their solution, don't talk to each other. Again you can use this idea of message bus and the common message format to get these systems to pass information from to and from each other.

What role does the bus play? You can do this and have a Java method that does this kind of scripting between operations?

Yes, but the bus, apart from providing this sort of persistence transaction and sort of scalability requirements that generally these systems require. It also provides a unified communication channel so that's the key here. But I must stress: it doesn't have to be messaging; a bus can be in-memory, it can be a file system if you want, it can be a series of http requests. Everything that arrives to the bus it must adhere to that common message format which is actually a contract.

A lot of people talk about ESBs and BPEL in the same breath. What is the relationship there?

ESBs address a lower level than BPEL. So ESB's address how do we mediate the message into a common format, how do we move the information around, how do we keep it transactional, how do we monitor it, how do we manage it. BPEL is a level higher than that. It talks about modelling actual business processes, so while we're delivering the information there are processes that actually act on the information and produce further events.

If you just have two services talking to each other, do you need an ESB?

Again it comes down to architectural choice. If you have only 2 systems and you know you're only going to have 2 systems integrating with each other, than you probably don't need the extra overhead of defining a common message format, serialising and de-serializing it into another application. However, people use ESBs when they have grander schemes about how they see their business processes being remodelled, streamlined and automated. So today you might integrate 3 applications on the bus, (let's take an example from the financial sector: we did a system that integrated six systems. However three years on they now have 71 systems on there. So that's where the value is. Is not in the initial two or three system integration, it's actually when you start moving forward and bringing all your other business system onto that integration environment.

What are some of the limitations of the ESB concept?

What we've just spoken about, having a common message format, I believe is also a huge limitation for lots of applications. What it boils down to, is that you should use something like a common message when it makes sense. But it makes sense only in a handful of scenarios. The biggest overhead of having a common message format is that you are serialising to XML. There's extra development effort with every system that comes in because you actually have to translate their information into a common message format. However one of the design principles of Mule was actually to eradicate the need, to have this common message format and in fact to eradicate the need have a centralized bus even. We do that by providing a very simple object to endpoint mapping that maps across the network. The information is sent along any communication channel such as HTTP or TCP or SMTP or JMS; the way it gets translated as it needs to when the service actually receives the message. So there's no common message format but there's a common set of meta-information that can be delivered with that message to provide further services such as routing and correlation and idempotent message receiving

What kinds of problems are too small for an ESB?

Problems that are too small would be the two applications. Two applications don't need an ESB and arguably even three don't, if you use something like Mule whereby you can go say we talked at this application via its web services interface, we talk to this app via JMS. And Mule will allow you to just route that with little or no coding. Purely XSLT transformation if it's an XML format.

What kinds of problems are too big for an ESB to solve?

It's not necessary size because ESB as a sort of architectural style or a model does scale up because you are bringing lots of applications on to a message bus and in fact there's nothing stopping you from having multiple message buses with an uber-message bus to manage them, so it does scale quite well. Where it falls short potentially is if you are bringing in all the systems onto a messaging system, the cost of actually converting into a common message format because by going through 20 systems is going to be quite expensive. You may want to look at other options to manage those interactions.

What's actually new about ESB? How is this an evolution for the message broker concept?

The interesting thing is that there's nothing new about it. We've been solving ESB-like problems using messaging for the last 15 years. IBM MQSeries has been there for ages, Tibco, etc. The difference now with ESB is that over time, practitioners have developed a number of methods and rules of thumb to develop these sorts of systems. And what an ESB does is encompass those principles and vendors have then productised those into a solution that they can identify as an ESB. Basically it is just the way of taking that concept and selling it.

What's the difference between an ESB and an integration broker?

In the ESB the central concept is the bus, whereas integration broker would probably be more hub and spoke so requests come into a single point which can be a single point of failure. The idea of the bus is far more resilient to damage because if one system goes down it doesn't mean everything else has to stop. You can always re-route or back a queue up until that system comes back up again. So it's really the decoupling that makes the difference between the two.

Why is the bus not the same as the hub?

Because a hub is a central system, whereas a bus is actually a data store, basically.

When designing with an ESB what's the right level granularity to expose a service on the bus at?

It's actually designed by the application or the service you're integrating on the bus. If you have an existing application, the granularity is the application. That's true for most existing applications. When you're integrating new applications on to the bus it may just be that you have a single web service that sits there, from another department. The granularity is purely dictated by the system that's being integrated.

Tell us about Mule.

Mule started about 3 years ago by myself. I've just finished a large integration project for a bank. I just felt there was a huge need for this stuff to be easier and I felt a framework could be built that could really change the way people build integration applications. Some of the core concepts behind Mule were to actually bring the cost of entry of integration right down to zero. By giving it open source people can try it out without typical costs of licensing people suffer with BEA or IBM.

What does Mule offer?

Mule offers a very simple integration model for doing enterprise applications. What that means is typical applications have a number of requirements such as security, transactions, scalability, failover and all service oriented environment. It also has the needs of talking over lots of different protocols and moving information around in a reliable and secure manner. Mule's goal is to do this and it has done it, but it has also exceeded itself because it has actually brought the concept of enterprise computing to the masses. It really simplifies the way you build these applications by just saying: you write the business objects or your services, you drop them into our container and you can expose them over many different transports and under the covers all the tricky stuff is taken care of for the developer.

Can you give some examples on how people use Mule?

It ranges quite a lot but I'll give you an example we currently have. It isn't an ESB, it's actually a sort of enterprise service network - lots of services connected using different communication channels. It's a large American franchise, they have thousands of satellite offices and their challenge is that they have a lot of information in these offices that they need to get to the head office so that they can centralise resource management and also keep track of intra-day sales and basically to provide day to day reporting. The constraints of the system are these: it is a big deployment so they can't afford a commercial solution. Open source makes huge sense, it's one of the clinching factors for them but they also need to be running their business so it needs to be a quality product. Mule provides both of those things. It is also very easy to deploy. It's a very simple, lightweight server that can be run stand alone, that can be deployed into these remote sites; the server itself is right resilient to failure. Short of pulling out the power supply even then it will come back up and still connect back to the network. We're working with them to develop the system and on the back of that once they have tracked the information, they can do a lot more stuff with this.

Can you contrast Mule to WebLogic integration or other integration solutions?

One of the key benefits of Mule is the cost of ownership. People can try out Mule, they can proof of concept and they can even go to production if they want without spending any money. That has a number of benefits. It allows people to validate ideas whereas in the past I've heard of a number horrific mistakes made by companies who probably couldn't afford to make the mistakes in the first place, by buying the wrong solution. Mule does that. It also provides a rapid development model because we don't' define any interfaces, we don't define any sort of restrictions on how an application is developed. You can write your Java Beans or you can put it in a new session beans or you can reuse your existing Spring Beans but Mule can manage it in its container and make sure the right information is passed to those objects. I think the key thing is the cost of entry and rapid development time. The fact that there's a healthy community behind Mule also helps a great deal. You can actually get help. We got a lot of people with a lot of architectural question on the list.

How did you pick the name Mule?

Mule actually came about from the initial project that sparked the idea. What I found was that once we built one or two adaptors to integrate these backend systems we were doing the same work again and again and it differed slightly for each one, but on the whole it was really the same thing. It was time consuming and I thought it was a wasted effort. One of the reasons for the name is that it takes a donkey work out of these systems. The other thing is that Mule is a carry over payload so the double meaning is that the Mule carries the message from one to place to the other.

How does ESB fit into the whole Service Oriented Architecture?

ESB is actually a small part of it. There's a lot more out there than just the ESB. Mule isn't just ESB. The whole point of Mule was to develop a framework for enterprise service networks which allow more flexibility in the deployment options. ESB is one part of the toolkit; there's also the web service, Grid computing, JBI as well and all those tools need to be used in the right way when architecting the SOA solution.