Adrian tell us about what you are up to these days?

Well, I'm up to a whole bunch of things at the moment. As many of you may now I've taken over the CTO Interface 21 and that's a very exciting thing to be doing,it keeps me very busy. I get to see a lot of clients, talk to them about Spring, the Spring framework, things in the Spring portfolio, still involved with Aspects and AOP, the AspectJ project and Spring AOP stuff, but also looking more broadly; where are we taking Spring framework, where is the portfolio going, how does they fit together. Leading the Spring OSGi project, in terms of technical excitement that's the really big new thing I've kicked off this year. Which again is quite innovative and doing something a little bit different to what we had in the industry and I like that. And that has been fun.

I did some training, I got involved in teaching some courses in Spring, teaching some AOP course, I put together a 4 days enterprise AOP; we take a lot of students through that. That's pretty rewarding for me, getting to sit down with a bunch of people for a number of days, be on the first 90 minute talk, show them how to use this to solve real problems; they love it and that's been good too.

What is OSGi, where does it come from?

OSGi has a long history that goes back into the '90s and it began for very light weight embedded applications, network devices etc. Where things would come and go we were doing embedded controlled software, like vehicles networks etc. And it was designed for a quite dynamic environment, very light weight, and gradually over the years it has grown to be used in larger and larger contexts. The latest Websphere application server is built on it. That's a long way from a mobile phone and embedded device but that's the roots of it. Over the years it has evolved and got picked up and now it's basically the leading state of the art micro kernel, there's nothing else to touch it.

So OSGi is at its heart a micro kernel?

Micro kernel is a term that is used sometimes, and a system purist wouldn't call it a micro kernel. It's a very light weight core container that understands what's the module in your system that wires those modules together and let's you publish services from one and find them in another. It's very light weight and small, and in that sense it's micro kernel like, it's not a micro kernel in the purest operating system sense of the term. That's the role it plays when you build a bigger server on top of it, exactly that kernel is holding the core of your system together.

How is OSGi being used today?

OSGi is being used in an increasing number of ways, probably the one most people are familiar with although they don't realize it, is Eclipse. It is totally based on OSGi. Any Eclipse plugin is actually just an OSGi module and in OSGi terms it's called a bundle, but that's all it is because OSGi module or bundle is just a JAR file with a certain manifest entrees. Because the Eclipse is getting very wide usage, one place is getting a lot of coverages inside, some of the IBM product portfolio, the web sphere the latest releases based on OSGi, some of the latest projects are building on it, number of other server vendors looking at it, as an infrastructure for future work, both big and small, definitely on the rise. Going back, it has been used in all kinds of mobile phone and light weight devices, but server side enterprise Java is coming on the radar, first from people writing server style platforms. Increasingly people say "You know what? That's an interesting framework for my application, because even my stuff is getting pretty big now. If you look at Spring applications, where they are getting over thousand beans and things are getting a little bit unweidly. It's fantastic that you can actually manage an application with a thousand beans but you gotta think that at 1000 beans there ought to be some internal modularity and some settings that could improve the structure of that. For those kinds of things OSGi offers something that looks quite attractive.

What impact could OSGi have on application development?

First of all think of it as a module system; if you glance back to my Aspects background and say "Aspects is all about modularity and OSGi is all about modularity", so there's a link for you. What it does is it actually properly solves first off the public private problem. Let's talk about it in the Spring context. I have some beans inside an OSGi module bundle, they are completely private, you can't see the types, you can't do any class loading trickery or reflection things, it is locked away. And then I can explicitly say, and I can export these types and these services and only that level is visible. Only that level is visible, but properly visible, you really can't get inside; and that has some important consequences like: suppose this module needs version 1 of some framework, some library to work, and it also has to work with this other partner that needs version 2 of the same library. Well normally now you're hosed, you can't get them both on the class path, one has to be first - system broken.

In OSGi that's not a problem, as long as it there's no sharing of those types across a shared interface. They can both work with the version they require concurrently. All those versioning issues and isolation stuff, the OSGi framework takes care of that.

First you get proper modularity then you get versioning concurrent deployment, and then the next thing you get is a level of operational control over that environment that you never had before.

Anyone of those modules, and Spring now says anyone of those application contexts we can install it, uninstall it, start it, stop it, refresh it, all at run time, keeping the rest of the applications up and running, without impacting it. OSGi had its foundations in, a light weight and dynamic environment, and it's been built to address the issues of that in a way that no other framework really has.

You get a pretty compelling set of benefits, and I just came from doing a 90 minute talk for attendants here at the Spring experience and there was a lot of excitement at the end and lots of people very interested in the possibilities of what it could offer.

Why is Spring interested in OSGi? What are you guys going to do with it?

I've explained some of the benefits that the core of OSGi platform offers. People say "Hey, can we have public private beans? What happens if we want to split something, not just into several applications files but into several applications contexts? How would I manage references between them? How do I basically structure the ever larger and more sophisticated applications that we all keep building?"

Spring has never got into doing its own complicated class loading and so on; it stayed well away from that, and we see that if you look at what OSGi has, totally leading micro kernel structure; its great module system support dynamics, it handles all that pretty well. It doesn't have anything that addresses programming model space or enterprise applications in the same way. Spring has all the rest of the puzzle but hasn't got a proper container runtime pieces. These seem to fit together really well, we would be mad to start building our own micro kernel infrastructure when we could build on OSGi, we never do anything that was better than OSGi, giving the time and resources.

The requirements we are hearing are: "I think OSGi would fit well with those". And it starts playing with it and says "Actually this is going to turn out quite interesting". And as it has gone forward, the level of interest in it has actually taken me by surprise, I didn't think as many people as they clearly are would be aware of an interested in OSGi at this stage, I thought I had to go out and sell it, and actually this thing has been dragged out of me.

We made the project public probably earlier than I would normally would, because of the entire level of interest, we got committers on from BEA and Oracle, very early on because they wanted to help us build it because they want to use it. Actually we're well over 300 people following the developers' mailing list without advertising it or especially doing anything just hanging on and seeing what we're doing. There's a lot of interest.

What do Spring and OSGi do?

What Spring OSGi does is basically saying "We've got all these bundles and framework but you still need a way inside the bundle to actually instantiate configure assemble and decorate its internal contents". That sounds awfully like a Spring application context. The first thing we do is we have a context loaded listener just like in the web space, for your servlet to stop an web XML application context; there's an equivalent in OSGi that when your bundle has started we'll automatically create an application context for it. Spring will manage all the internals of your bundle. That's the first key piece that we are doing.

But then OSGi also has like a service backbone, called the service registry; imagine we have now created a whole pier network of application contexts, and then they want to interact with each other, they might have one that has got your business services in it, one that as got some data access objects, they are going to be separate bundles maybe, but you need to have references between them, the services need to get a hold of the data access objects. How does that work? Spring basically solves that problem as well. You can take any bean and you can trivially expose it as a service in OSGi and then in the other bundle you can easily inject a reference to an OSGi service and so we can wire references, between application contexts using the service backbone and that's done in a way that copes with the fact that this service might go away, or another one might come along, or the bundle might come down, it could be restarted it could be refreshed. And Spring manages its best it possibly can with all those dynamics; we basically give you a constant proxy that you work with and behind the proxy we manage in the target source.

What does Spring OSGi enabled application to look like? What are the moving parts?

Basically bundles sound very grounge but they are just straight forward JAR files, there isn't any sophisticated packaging etc needed. Infact the minimum is this: there's a JAR file in it having it manifest.mf, an entry bundle symbolic name and you give it some name. That's the absolute minimum you need to do to become a bundle. You will probably have a few more things in there like these are the packages that I require or import in OSGi terms "this is what I provide or I export".

So have a bunch of JAR files with those headers in them. And then inside those bundles how does Spring know I've got to create an application context etc so what we do is by convention in the metainf there's a subdirectory called Spring, inside there we'll take all XML files that you place in there and say "Let's build an application context using that as the configuration". That's the default rule, you can actually put a header in the manifest say "Use these files" but default will just find them all. And what you are ending up with is a series of OSGi bundles that are JAR files that have a certain set of entrees in metainf, it's enough for OSGi and Spring to do their work, that then has to be packaged up into some OSGi runtime. Normally that's just a simple matter of dropping these JAR files in a certain directory and telling OSGi "This is your bundles here, repository location, and these are the ones I want you to start when you start up". It's "What you see", that's the basic packaging structure. It is actually fairly straight forward.

What is there in the areas of testability?

Spring has always placed an emphasis on the ability to unit test, integration test, test outside of the container, and in general to make sure that you've got components in isolation.

One of the things Spring and OSGi do is takes away the need for you to depend on OSGi APIs so you're back to simple objects; Spring is going to manage all the OSGi interactions for you for a very large degree. That means the classic "we can unit test those components etc". Clearly we can integration test with just the context as normal. Then what's more interesting is that sometimes you want to say "Well how does this behave in OSGi when I've got services wired across contexts, is it doing the right thing?" That's a non trivial problem to start, running like JUnit test either from your IDE or from your continuous integration built inside OSGi and get the results out, and we scratched our heads for a while and we ended up building a little Unit testing framework that supports that. Like when we had the Spring mockup with the whole bunch of classes that you can extend from. Actually we have a couple of abstract based classes you can extend from that run OSGi tests. And what they do is actually quite smart. You kick off the test and you say "These are the bundles I'd like to use for this test". And the test case itself when it starts up will boot strap your OSGi environment and it can do either Equinox Felix or Knopperfish based on the property you set. It boot straps OSGi installs the bundles you asked for that you need for that test. Then it creates on the fly a virtual bundle out of your test case, because the test itself needs to run in the OSGi environment, installs that virtual bundle into the OSGi that's created for the duration of these tests, fires off the test in the copy that's running inside OSGi and funnels the results back and gives them to you. So it's actually possible to test from a straight JUnit in an IDE or a build and get OSGi kicked off and run all these. And now you can do test like "Ok, did the application context start, was this being exposed as a service? Yes it was, you can verify that". Take this bundle down, have I still got references, all those things you can now test easily from JUnit. I think we can still get even better but the support it's a lot better than it was before.

Given that OSGi dynamic, modules can come and go. How does Spring handle a situation if a module goes away?

You've got a number of application contexts, you basically got references across them through the service backbone, and what basically happens is the you get a reference to a service or set of services, you specify some properties, normally the interface that you are interested in when you do the service look up. And it could be that at that time there is zero, one or many matching services, and depending on what you specify as the cardinality you are interested in, there are different rules.

If you say "I just want an implementation of this service", then OSGi platform has rules for which one will get picked and you'll get the service. If you say "I'm interested in all the matches", Spring gives you a set or a collection. Behind that Spring would transparently manage the members of the sets. So you can iterate over it to call or whatever you want to do. The current members will be whatever services are available, that's managed transparently.

In the case of a single reference what you get is a proxy that you work with as a constant reference. Behind that Proxy, Spring is tracking the service that you are currently using and other services that may become available that match. So it is possible for the backing service to go away and come down; if you don't use it in the interim you won't even notice that. If you do use it when there's no valid service around to provide a match by default, at that point you get a service unavailable exception, that is possible to say "you know I don't care this is like a one way deal, Spring will just call it if it can, or not."

But you get some interesting scenarios like "I'm working with a version of a service and I now need to upgrade that service I'm dependent upon". It's possible to bring up a new bundle that provides the new upgraded matching service. I've got a client, I'm under heavy load, I'm under stress. I'm working with an existing service. Take down that existing service I was working with behind the proxy. The locking semantics, the way that OSGi guarantees we get call backs about this, we are actually able to map you over onto the new service that exists and the client will never see a glitch, it will just cut across.

Obviously that works best when the backing service is stateless. If you had some state between you and the client there's not going to be totally transparent and in that case it's possible to ask Spring to call a bind or an unbind method, so that you can be told when the backing services are switching and then you can reestablish any state etc.

Basically the answer is Spring is tracking the services behind the scenes and trying to smooth the waters as much as possible, using either a straight proxy or a proxy collection. And it deals with all the dynamics under the covers for you.

How do you see enterprise applications using this to build more modularity in their code basis?

Initially I thought the prime market is that you see clients and they are building big stand alone applications that do back office stuff, they need a way to structure those and this is like the perfect foundation for them. But actually there's strong interest in structuring large web applications this way; the size for some of the things that people are building are getting pretty complex; we have the classic layers, if you look at the way Spring will do it. Yes you got your middle tier application context, then we create the web one as a child of that and you get some visibility constraints around it, but that's only object visibility, there's only one class space there, that's the problem. And also just for sure, when these things get large they have vertical partitionings as well.

What a large web application will look like in the OSGi world probably 1, 2, 3, how many MVC bundles, so take each layer and partition it vertically, so that you got the side by side partitions of the system. Then in your service layer, again divide that by major service root; each of those would become a bundle, exposing its services. The same for your repositories; the domain main model might be split across one or more bundles. The supporting libraries you need - they are also bundles. And Spring OSGi is going to wire that together for you, using the OSGi service registry to do the cross bundle bean things. When you got these larger complex web applications, suddenly you get a way of getting a lot more structure and control around it. When you've got those big large web applications, you probably got a big large team working on it. And it's going to be much easier to update and extend parts of it separately, in parallel, and it wouldn't have been previously when it was one big lump.

Why modularize an app using OSGi bundles versus separate WAR files?

If there's truly no interaction at all across the separate pieces I think separate WAR files is a better bet. It's going to be a simpler technology, but generally you have got some shared parts like maybe they want to work with the same domain module, you've got one piece there, but you get better partitioning in the service layer, and if you end up with one big general WAR file then OSGi starts to play. If you can do something simpler that works then do it. Right now today you are on the bleeding edge if you try to construct a web application with OSGi; we used to have a lot of work to do to make that smooth.

I'm trying to think when within an application would it be a good idea and when wouldn't it be a good idea to do that, to use modules that way? I mean within a trust boundary of a development team? Why would you want to hide the user management sub-system?

Why do you use packages? It's the classic "What's the benefit of modularity?" question. It's all about enabling those things to evolve a little bit more independently, making sure you don't accidentally couple to in it, the classic problem about things being public, simply because you want to go outside the package versus do you want to go outside the module it will deal with all those issues. Even from the "why do we have classes" questions, that's about the same to "why do we have packages?", "why would you care about groups of packages"; it's the same issues as we move up in scale and the bigger, more complex thing you need to do, the more interesting tools you need to deal with that. If you can get away without it in a simple case, then do so.

Who do you see as being the main users of Spring OSGi?

At the moment I think the main users are people who are trying to themselves provide service platforms, those are the people who are most active and vociferous and interested in it. Frankly because if you are trying to build web applications, or back office applications in this way, we haven't got the thing fully baked and packaged out with all the getting started guides to enable you to do it yet.

Right now there are people interested in building service and service style platforms; I think it's going to be second half of 2007 at the earliest that we get more serious interest in "hey we can use this to build our own apps" But when we show the capabilities to people that have just been doing that, a lot of drooling and salivation about what might be possible here but it really packaging up to make that totally easy so that some of the OSGi is hidden and you don't have to think about packaging these bundles, and we don't break round-tripping and all those nice issues, that all has to be smoothed out, particularly in the web application space; we got some work to do there early next year to get that right.

You know how JBI to an ESB provides a way to plug in separate implementations of different parts, could OSGi provide that to the notion of application server?

OSGi absolutely gives you a foundation in which, like you have seen with Eclipse plug ins, you have a framework and it's very easy to dynamically add and remove modules as you are going along, to extend the capabilities of the platform, to do that at runtime. That's the kind of thing that OSGi was built to do from the very beginning. Yes, absolutely can help you build platforms like that.

How does OSGi change the definition of an application server?

To some extent that depends on what the application server vendors choose to do with it. You can exploit OSGi in a number of ways: one is basically "hey, I happened to use it internally but there's no real way you could tell; it's just my private way of partitioning out my internals" or you can start to do more interesting things like actually use that OSGi foundation to enable the kind of stuff we have just been talking about: "hey, here's a way to easily get plug ins to the actual based server itself and to extend its capabilities". If you start to expose that sort of thing it's going to get quite interesting because you are really allowing your users to customize their platform, the major components of it in a way that we haven't really seen application servers do to that degree yet. That would be technically possible to get to that kind of level. Whether they choose to go that far? We'll have to see.

The community seams to be moving away from waiting from this next monolithic J2EE specification that says "here is all the things an application server should have?"If OSGi becomes that popular, could there be the expectation or could become the norm to the kind of assemble your own subset of an application server, where different parts evolve separately like the EJB can evolve separately than the JSP portion, and you also wait for this monolithic umbrella spec anymore.

Technically that's the kind of thing that OSGi enables to happen the question now becomes: will the marketing organizations and the sales organizations think that that's the right thing to do from a business prospective, but it puts options like that on the table and I think the market will be receptive to it. I think the big monolithic "let's wait to years and years, and here is that whole big new thing", we kinda got past that. People are much more used to moving quicker than that; they still want a stable trusted platform, but I don't think that they necessarily want to wait 3 years for something that was specified 2 years before that. OSGi would give us that kind of flexibility. People will take things out as well.

Could Tomcat become basically the stable base you want and things get added to it?

Yes, it would be entirely possible for something like Tomcat to support the OSGi foundation; in fact if you look at Equinox they already have a Jetty bundle inside that can start a Jetty container for example. You run all that inside of a OSGi so it is entirely possible to do some interesting things like that.

What's your take on JSR 277 vs. OSGi?

I think the controversial one is 277 which I think is entitled the module system for Java. What 277 does is a static subset of OSGi capabilities, it gives you the equivalent to what in OSGi is called a required bundle so you can say that "I'm a module, and I need these other modules and you can name them explicitly". It's actually a mechanism that is slightly frowned upon inside the OSGi community, who'd prefer a more granular imported based on package and exporting based on package that allows you to mix and match providers.

Because you don't really care what bundle the module provided this capability you just want to say what capability you want. The other thing that's missing from 277 is any notion at all that could be dynamic things might come and go.

That's kind of hard to retrofit; most systems come along and tried to retrofit it have struggled. My favorite place to really understanding this is Peter Kriens, the OSGi technical director. He has a blog entry which is a pretty extensive critique called "what' wrong with 277…why didn't they take advantage of all these years of experience that we've got." To paraphrase him: the expert group have completely ignored 8 years of experience and made a number of serious mistakes. 277 is interesting but let's see what happens to it. At the same time there's another JSR which basically says "let's take OSGi platform and make it available in Java" which is clearly in conflict with 277. How will that pan t let's wait and see, I hope the OSGi based JSR wins and the other one dies away but you never can tell. In the meantime I think it matters less and less because we have a stable working proven OSGi infrastructure that solves these problems that we can use now without waiting for expert groups to decide what they think we should do in a few years time.

What can you tell us for the future given OSGi?

What I'm most excited about for the future, I think it's just that right now we understand the tantalizing set of possibilities, and we can see some things that look like they would be very interesting. What we really don't know is how is this actually going to be rolled out for real and real applications, what are going to be the patterns, where is it going to give benefits to and where not. Seeing an incredible rise of interest in people I think we're going to get answers to that question next year. And if it does pan out and if it starts to get used and we get this much much richer operational control of applications that could be very interesting world. It's going to basically change the deployment and run time model for enterprise applications and that would be scary, exciting, interesting, everything all at once.

Adrian, what are your 2 favorite computer books?

I really like and I keep getting back to it "Structured interpretation of computer programs", a real mind blower every time you return to it. That's a good one. Second, I think in terms of influence actually it's an old one, and probably a surprising pick because it's not theoretical; if I think back through my career in the earlier days, a book that had a real influence on me, and still has a number of tips that I use day to day is "Rapid Development", by Steve McConnell. It's a really interesting inside stare into the process of managing software projects, it's an oldy. And it's not theoretical at all but it has basic advice on sizings and metrics and practical tips. That had quite an impact to me right on the start of my career. I haven't gone back to it for a long time, but it's an interesting one to highlight.