Oleg Zhurakousky on Messaging, Spring Integration, and Cloud Architectures

Recorded at:

1. Hi everyone, my name is Srini Penchikala. In this interview we’ll be speaking with Oleg Zhurakousky. Hi Oleg, can you introduce yourself and tell us what you are working on currently?

Hi, my name is Oleg Zhurakousky, I work for SpringSource VMWare and I’m in the engineering group and I’m a committer for the Spring Integration Project and contributing to some other Spring projects.

2. You recently spoke at the JavaOne Conference on building the cloud-ready systems using open technologies. Can you tell us what technologies you’ve discussed in the session?

It’s a good question. I have talked obviously about Spring Integration as the binding framework and I’m sure we’ll talk about this a little more in further questions, but, as you noticed, the topic did not have the name Spring because I wanted to cover not just the Spring technologies but some of the other technologies that Spring integrates with and that I also believe are important for the Cloud development, so I was covering RabbitMQ, Redis. These are all new technologies and that Spring is now starting to integrate within the past year.

3. It looks like there are definitely different technologies available for Cloud development. Speaking of technologies, can you tell us what are some technical challenges for organizations to adopt the Cloud?

We no longer live in that world where we could rely on using a single vendor system. There is no "one size fits all" anymore, because the Cloud itself is giving that level of transparency. So we have challenges with resources because in the Cloud I want to have access to many different resources, not just for example dealing with the backend data that I have - so it’s RDBMS, key value stores, messaging systems. When it comes to the Cloud, we need access to pretty much all of them. As I mentioned earlier, the platforms - I mean WebSphere, Tomcat, WebLogic - we no longer want to be tied to a particular platform, we want to think a step above and say "OK, if I want to do JEE Development, then I just want to be able to have a Cloud platform that allows me to deploy JEE applications. If I want to do alternative approach and use Spring, for example, then I want to be able to have my clouds support that as well." So it’s really becoming more about the programming model than a particular specific product.

Scalability - we came from the world of clustering servers and that itself is becoming a legacy notion because maybe that’s too heavy for me, maybe all I need to do to achieve some level of distribution is distribute some of the processes or some of the values between the processes. So in other words, I want to be able to accomplish that and some of the frameworks that Spring provides allow you to do that, but I no longer want to venture into a proprietary scaling model provided by a particular vendor and then if I, God forbid, change a particular platform, then everything else should change. - No, it shouldn’t.

Some of the other application developer challenges is information availability because back in the days it was either SQL to get data from the backend or JMS or something along those lines, but right now, a lot of different data is becoming available via SOAP, REST, HTML, JMS, AMQP - you name it. So there are many different protocols that give you access to the data and you want to be able to integrate with all of them. What is the best way? Is it REST or is it SOAP or is it JMS? My answer is: there is no "best", it’s whatever works for you.

I want to be able to give you access or give you the ability to do pretty much anything you want, so there shouldn’t be, in my view, any limitation. Then, obviously the pervasive devices - we all have iPhones or Androids or some things like those and we’re pretty much married to them, so we want to be able to get the information quickly and access it through those devices, so we need to be able to integrate with that as well. These are some of the challenges that we have when it comes to the Cloud.

4. That’s a good point. Information availability is very critical right now with the mobile computing, social computing, and the new trends.

Speaking of social computing - as you probably aware of the Spring Social Project - that’s really what it’s all about.

5. Cloud computing is not just a technology related concern, it’s definitely more than technology. What are some of the non-technical organizational challenges for the Cloud adoption that you see in the organizations?

A non-technical challenge to me would be understanding what Cloud is. For example yesterday, when I was presenting at my talk, one of the questions that I asked them is "What is the Cloud?" And before even getting an answer, I said "Let’s play a game: if I ask every single person in the room, ‘What is your understanding of the Cloud?’, what are the chances that we are all going to get the same answer?" and the answer was "Zero". How do people understand the Cloud? How do organizations understand the Cloud? There is no specification, there is no standard around. It’s such a new technology right now, it’s such a new buzz word that all of us are still trying to figure out what it truly means and that’s why there are many different "Cloud platforms" but you look at them and some of them are quite different because my understanding of what the Cloud should be is different than your understanding of what the Cloud should be.

Eventually it will evolve into something substantial and more stable, but right now it’s very unknown. We know that we need it, but we don’t really fully understand what that really is.

6. It’s a good point that Cloud computing is such a broad topic because each organization has to define what Cloud computing means to them rather than someone else. Can you talk a little bit about how different Spring frameworks can be used for architecting for the Cloud?

Before we even talk about these frameworks, I would like to mention that frameworks help you to be a better developer, but before you even start using the frameworks, there are many different frameworks out there, so you have to choose the right ones, but to choose the right ones means that you have to choose the frameworks that fit your architecture and so the real question here is (before we even get to the frameworks) "What architecture would you choose when it comes to developing for the Cloud?"

I’m a big fan of messaging and a common misconception of messaging within the Java eco-system, is that every time I mention the word "messaging" developers obviously start hearing JMS and that’s not what I’m talking about. Messaging is an architectural paradigm that allows you to architect your system by exchanging messages between the producers and the consumers. That does not imply any type of remoting between the producer and the consumer, it just simply implies decoupling of producer and consumer in such a way that, if at some point in time, for example if those producers and consumers have to be distributed, then that medium that exists in-between called "destination", "channel" - whichever you wish to call it - could be swapped through implementation that uses different protocols.

For example, if you are using a framework like Spring Integration, by default you will be simply exchanging within the same JVM process by reference. If you swap that channel to something like JMS or AMQP then, all of a sudden, you achieve that remoting without making the producer or consumer aware.

Now we got to the concept of the frameworks and Spring Integration, the framework that I represent, is a messaging framework that allows you to or helps you to develop systems and applications that are architected in a messaging style. Like I said, I believe it’s extremely important right now to start to resurrect the concept of messaging instead thinking in messaging because in the Cloud, we no longer have the guarantee of a single VM process. Basically I’m deploying something at some point of time, it might spin off another VM. So I don’t know how I can adapt if I know your system is deploying the messaging using messaging paradigm then, I as a framework developer, can adapt to that level of distribution so that it’s transparent for you, but if you are relying on a direct by-reference call and all of a sudden that thing became remote, then you as a developer can no longer rely on something like that.

Spring Integration and Spring Batch obviously and again there are a slew of Spring products like Spring AMQP, there is a lot of development going on right now around asynchronous messaging within various different new open-source Spring projects - so we are really seriously thinking about this messaging paradigm that it is the true architectural style that you have to use when it comes to the Cloud. There are competing frameworks outside of Spring which again are competing, but there still represents the same idea that you should utilize messaging if you want to achieve that level of scalability, that level of distribution and that level of transparency that the Cloud gives you.

7. Spring also has Spring Roo as well as Cloud Foundry frameworks. Do you want to talk about them in the context of Cloud?

Sure. Spring Roo is obviously a productivity tool, that’s what helps you as a developer to get things done quicker, especially if you are starting something new. At the very beginning, Spring Roo kind of started with trying to get your basic Spring MVC application up and running, but it evolved pretty quickly and right now there are Spring Roo plugins and Spring Roo support for deploying in the Cloud. You mentioned Cloud Foundry - Cloud Foundry is the product from VMWare and in conjunction with Spring Source this is basically our version of the Cloud and our understanding of the Cloud. Spring Roo allows you to develop and deploy application quickly to the Cloud Foundry and that’s pretty much it.

8. It’s good to see all these frameworks working together, Cloud Foundry being the Cloud computing solution and Spring Roo is the development framework and Spring Integration is the messaging middleware. You mentioned about JMS with the asynchronous messaging and event-driven architecture being the critical elements of Cloud architecture. How do you see Spring Integration framework helping developers in the Cloud?

It’s a very interesting question. With all this new technologies and different products that come for example into the Cloud Foundry or as I was mentioning before, that you have to start thinking about working with, as a developer you really have to do a lot of learning today and sometimes not all of us have the luxury to all of a sudden, overnight you have to learn Redis, Mongo, RabbitMQ and all the other stuff. Frameworks like Spring Integration provide you an easy way to integrate with those systems without you (unless you choose to) get into the complexity of it. For example, if you know that, if somebody told you that "We are going to be integrating with the other systems via AMQP", you have two choices: you can use raw API and by doing so you would have to start learning a lot about AMQP concepts or you can skim the documentation, understand some of the basics and say "Ok, Spring Integration allows me to configure an AMQP adaptor - inbound or outbound - and all of a sudden all I need to do is just to send a message and I will take care of all the protocol details, all the failover things and so on and so forth." And the message will get to the other side and you’ll get a reply back if you expect one.

The same goes for Mongo, the same goes for Redis and other things or JMS for that matter. Now, all of a sudden, as a developer, you can concentrate on what exactly you are trying to accomplish without worrying about some of the low level details. It basically brings that Spring model where we’ll take care of the boilerplate code while leaving you with only what your use case tells you to do. It brings it to the messaging in general. Spring Integration is like a binding framework in that way where there are different technologies, that are coming right now into Spring and obviously into the Cloud. We provide integration points with as many as we can at this point. For example, currently we have support for Twitter, Mail, RMI, FTP, SFTP, AMQP, Redis, Mongo and I can go on and on going on. All of a sudden, all these different protocols are available to you and integration with every single one of them would be the same because we abstract it into these abstractions that are defined by enterprise integration patterns and all you need to do is understand how to send a message to it or how to receive a message from it. And it’s the same for each and every single one of them.

9. With all these new technologies comes the need for developers to have some tools support in terms of testing these different components in the unit testing phase as well as troubleshooting and debugging applications at runtime. Can you talk a little bit about what kind of tool support is available right now for working on these Java applications?

Obviously everybody knows we have Spring Source Tool Suite, so it’s obviously Eclipse-based development environment tailored specifically for Spring, so there are a lot of plugins that are specific for Spring projects. There is a first class Spring Integration support which not only allows you to write and code and develop your Spring Integration-based project, but it also gives you very different visualization tools to allow you to envision what have you assembled, what flow have you assembled and how it’s going to work, because that becomes quite important especially when it comes to messaging because at the end of the day it all becomes boxes and arrows and navigating the message flow and see where it’s going to go.

Obviously, you mentioned earlier Spring Roo and I want to mention something else like I am also working as a developer for Spring Integration Scala DSL. We started this project for the reason that Spring Integration is a Java framework, but we want to be able to tailor to many different communities and Scala is a new language, but Scala is still a JVM-based language, so it could easily integrate with Spring Integration, but if a Scala developer doesn’t like to develop in XML, to write configuration in XML, why not provide him a simpler way to use his native environment such as Scala to develop. We’re talking about providing the same capabilities for Groovy. I actually just had a discussion with a couple of our guys downstairs about that, so we are kind of exchanging ideas.

These are all things that are coming up shortly and will be available to the developers. So we are very seriously thinking about community-driven and listen to the community requests and these are our responses, so we are all about the tooling and make your life as simple as it could possibly be because it’s already hard.

10. You mentioned DSL. I think DSL is a better choice for defining the routing and the other integration component dependencies, more human friendly.

It’s not just the human friendly. I was just talking to Hans Dockter, the guy behind Gradle, it’s not just about user-friendliness, it’s also about the fact that, you know I mentioned a visualization tool for Spring Integration - that was developed for us because there is really no native support. But if you are developing something using the native language, for example I have installed a Scala plugin for my STS and right now I don’t have to create any special tool to actually start writing or do a code complete and things like that because it already comes with the fact that I have a Scala plugin that recognizes Scala. I now know that, without any additional involvement from other development environments, for example, if a guy wants to use IntelliJ and he wants to use Scala DSL and IntelliJ provides a first class support for Scala, then they don’t have to worry about Spring Integration or other projects because it’s really not about that project, it’s really about Scala. The same would be for Groovy, the same would be for other domain specific languages.

11. You are also working on Spring Roo add-on for Spring Integration. Can you share with us what is the current state of that? Any timeline on when it will be available?

I can’t give you any timelines unfortunately because right now there is just so much going on and our priority has slightly shifted. So it’s not dead, it’s out there, it’s in a state where people can try and play around. I always encourage people to go play around because right now it will not generate a lot of stuff for you but it will give you a feel and we are also looking for feedback, because when it comes to development tooling the vision that I might have how a tool is supposed to be, is meaningless unless the wider community has embraced it. In other words I don’t know whether the community shares my vision or not. It’s out there, people can play around with that and see how it feels and whether it feels right, whether it feels wrong and I’d be more than happy to incorporate the suggestions, but right now, I would say that the status of it is on hold, but it’s not dead, certainly.

12. Final question, what are your favorite IT and non-IT books?

The IT books - there is only one for me, we almost call it that’s our Bible, that’s the Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf. That’s pretty much our specification for Spring Integration, that’s the book that’s constantly on my desk and I constantly refer to it. As far as non-IT books - I unfortunately don’t have a lot of time to read non-IT books right now, but back in the days and a lot of times if I do get a chance to, if I do find time, I’m a big fan of Sherlock Holmes books. I’ve read them actually several times all of them and every now and then I try to re-read them because I like those stories.