Project Lead Mark Fisher Discusses the Spring Integration Project

1. Hi. My name is Ryan Slobojan and I am here with Mark Fisher, the project lead of the Spring Integration project. Mark, what is Spring Integration?

Spring Integration is a project that was originally inspired by the "Enterprise Integration Patterns" book written by Gregor Hohpe and Bobby Woolf and basically the realization that those patterns could be implemented and supported in a framework that is built on the Spring programming model. So we really view it as a natural extension of the Spring programming model, adding a layer on top to handle those integration patterns. So you can do messaging inside of an application using patterns such as routers and splitters; even if you are not connecting to other external systems you can do that intra application messaging, but then the other side of the Spring integration project is adaptors that do connect to external systems, so you can add file polling, JMS support, web service endpoints, all of those types of things that basically just connect to these message channels within the framework.

2. One of the things that I have heard mentioned is that Spring embodies Enterprise integration patterns. What are those and how does Spring Integration help with those?

The enterprise integration patterns actually cover a pretty wide spectrum of anything that you would need to do when you are building event driven architectures that use messaging as the means to communicate between systems. So you have things that are as simple as the message itself is basically a pattern, message channel is a pattern and then there are different kinds of channels like point-to-point, publish-subscribe. Then the next category of patterns really are the endpoints where you have things like routers and filters and transformers (called message translators actually in the book) and channel adaptors and gateways to connect to other systems. So these are all patterns.

I think what is interesting is that a lot of frameworks in the space talk about being an implementation of the enterprise integration patterns and I think that is true to some extent with Spring Integration. Some of the patterns are things that you can implement, like we have an implementation, several different implementations, of message channel; other things are more conceptual, like there is a pattern called "request reply" and that is really a way of interoperating with other, composing something out of multiple patterns. So there is that part as well, things like the return address, which is implemented as a header that you store in a message and then deal with down stream. So it's an over simplification I think to say that it's actually an implementation of the patterns. Some of them can be implemented, others are just addressed by the framework.

3. What are some of the different systems which Spring Integration currently supports as an integration endpoint?

The integration endpoints that we support in the 1.0 version of the framework are JMS, File system, HTTP, web services; we have support also for Spring application events which allows you to actually hook into a Spring context and we have some support for OSGI as well, so that is a slightly different category, but being able to export your channels and import channels within the OSGI service registry basically.

4. What are some of the things which are going to be arriving with the 2.0 release of Spring Integration?

In 2.0 the very first thing we did was start to build on top of Spring 3. So there are features that were available in Spring 3, such as expression language support is going to be a really big one. We've also started exploring direct scripting support with Groovy, so even though you can today have a Groovy script that is loaded basically as a Spring bean and reference that from something like your message transformer, what we are trying to do is build it in directly, so I can reference a Groovy script and that script can be as simple as necessary. It doesn't have to implement an interface or anything like that.

We've also added an AOP interceptor so that any method that is invoked can actually fire off a message and the way that you construct that message is also using the Spring expression language. So what you put into the payload, what you put into the headers, can come from the method invocation itself. And we've added a number of adaptors, so in 1.0 we were focusing more on the core programming model itself and the core patterns as opposed to providing adaptors for every different system out there. We basically focused on mainstream adaptors to do things like JMS and the file and the web services.

In 2.0 we are starting to expand that out a little bit, so we are adding things that are still considered mainstream within the core, like JDBC, RSS feeds, we are going to add XMPP support; you saw the preview of the Twitter support in the keynote last night. And we are also trying to really promote the extensions project we have where people in the community can actually contribute adaptors that plug in. We've tried to make it as simple as possible to have inbound and outbound adaptors, it's literally just a method call, so you can actually build a message within one of these adaptors and we are hoping to get a lot more community contributions for all those things that kind of fall outside of the mainstream.

5. You had mentioned that Spring Integration uses aspects to fire off events and there is also existing support for events within the Spring framework. Can you kind of describe how these two relate to one another?

Sure. Actually it's a really good question because people have asked that from the beginning of the Spring Integration project; how this differs. With Spring 3.0 I think things become even more subtle because in Spring 3 the application listeners are now parameterized. So you could have an application listener that is only interested in a specific type of event, some other publisher that is pushing those events and then you kind of have this connection.

So basically you could implement something that is doing that, where you have your own aspect that you write, and that aspect gets a hold of the application event publisher and fires the event. The difference with Spring integration? Number one we are going to provide that aspect out of the box, so you can add the annotation on a method, use the EL expression within that annotation to fire the event or you can put all that in XML.

And really the main difference is that the message is then going to a channel and you can tell it explicitly which channel. That way you have subscribers that are only interested on that channel. So it's really a matter of scalability; instead of being tied by the type of the message you have this channel that you are connecting. So I might have 20 different places in the application that I am firing off events when methods are called, but each one of those goes to a distinct channel, some of those channels might have multiple subscribers, some might have a single subscriber. You basically have the point-to-point versus publish-subscribe channel options there.

6. How does Spring Integration compare with ESBs?

I think the main difference between Spring Integration and any ESB is the fact that Spring integration is embedded within a Spring application context. So you literally add a channel and that just adds another bean within the context. You add an endpoint connected to that channel and that is another bean within the context. So it is very easy to incrementally adopt this and you could run that application context in any environment.

So one of the big challenges in the original design is how to do this without actually even subclassing the application context to have our own specific type, and since we solved that problem, you can actually run inside web application context, you can run inside an OSGI environment, you can run anywhere that Spring can run. It could be within something like wait(), it could be a standalone application that you kick off from a main() method, it could be a Tomcat server, it could be an application server. Whereas with ESBs I think that's the main distinction, is that you are deploying your application into that ESB, the ESB is the runtime. Even light-weight ESBs that might interoperate with Spring, it's still kind of a side-by-side solution where there is some form of lookup of the Spring object from the ESB as opposed to the messaging patterns growing out of the application context like they do in Spring Integration.

7. What are some of the examples that you've seen of Spring Integration being used out in the wild by the community?

It's interesting because you can apply these enterprise integration patterns to a lot of different problems out there. So you see a wide variety of applications. I think one thing we are recognizing is that a lot of people have built these sort of frameworks in-house for their back-end processing, so not specifically in the web application environment, but when you are doing batch processing and things like that. So we are starting to see applications that use Spring Integration and Spring Batch together which I think is a really powerful combination. It allows you to have messaging that drives your batch job and messaging as a result of a batch job, when it completes, when it fails, when something goes wrong inside the job you can fire off a message.

That is something that Dave Syer and I are going to be presenting tomorrow in our session. Another area that we are seeing people using Spring Integration is with OSGI and the Spring DM project and usually deploying into the dm Server where you have this natural fit of the pipes and filters model for enterprise integration and then the modular model of deployment with OSGI bundles. So for instance you can have a publish-subscribe channel and you can add and remove bundles within that application that become new subscribers or subscribers that disappear as you take those bundles out, and that kind of dynamic modularity runtime - the kind of hotswapability of bundles - is really a good way to use the framework, I think.