Sanjiva Weerawarana on Open Source SOA Middleware

Recorded at:

1. This is Stefan Tilkov at QCon San Francisco. I'm talking to Sanjiva Weerawarana. Welcome! As usual I would like to start with a question, can you tell us a little bit about what you do

Sure, I am CEO at WSO2. I founded this company two and a half years ago now, and the company is basically an open source SOA middleware company. We are trying to figure out what the right middleware is, and build it.

2. Can you give us a little bit of history about yourself?

Yes, my background in web services goes back to the beginning of web services. I used to be at IBM research for eight years and in 1999 I got involved with the effort to respond to SOAP basically from IBM. And I was doing something called BML at that time, which was a component composition language, which I had worked on. And through that I got more and more into this XML service, remote execution functionality and then we ended up doing an implementation of SOAP, called Apache SOAP, IBM SOAP4J which became Apache SOAP, and then I ended up working on WSDL. I created WSDL, which is something not everybody loves all the time and went on to do a lot of specs, most of the web services specs basically, I was part of the BPEL team and part of the team that negotiated with Microsoft on a lot of the specs, as well as a lot of open source implementations as well as IBM implementations, and in the process of doing this in 2001 I started a project in IBM research, called the Colombo project because I moved back to Sri Lanka at that time, which was basically saying that the way IBM was building middleware for web services was not right and there was an opportunity to do a better job by starting again from scratch, from the JVM and building a new platform. I did that for about two and a half years, built a complete prototype by the middle of 2004, I took it all the way through to IBM and it wasn't in the direction that IBM wanted to go at that time, because it was a very J2EE centric world, and that's when I decided to leave to form this company.

3. Given that you've been involved with web services for so long, how do you respond to some of the criticism that surrounds it at the moment?

I think there is both fair criticism and unfair criticism. The fair criticism is that the space, if you are an unknown person trying to learn what web services mean, and you google for web services and you are looking for WS-Something, you are going to get a whole bunch of stuff. And that's an unfortunate situation but it's a reality in the way in which standards were defined. There was a lot of inter-company politics, there was a whole bunch of stuff that happened, but really at the end of the day there are only 10 specs that are really core to web services. If you get past the hype and if you get past the unfair criticism, in fact there is a good core. It seems a little bit unfortunate that there is this unfair criticism, but on the other hand the web services guys are also to blame because there was a lot of time when the web services guys basically, first of all there's a camp in web services that certainly did come from CORBA, and it's about object people who basically try to reinvent the distributed object system with XML. The great global WS-RF grid forum basically ended up doing that. And that gave all the ammunition to the light weight community to say "Hey, this is crazy, the Web is not an RPC system, that's not what we should be building". And because the web services community was a bit far right so to speak, the REST community became far left. Now I think as we go forward we have to figure out the right medium. I don't think either one is completely right, I don't think completely going all the way to REST in every single thing that can be built with the REST system is right, I think there is some middle point, there are applications which have a natural fit in web services and some which have a natural fit in the REST style approach. And trying to shoehorn one into the other is not going to make it.

4. I'll just note that it was you who brought up the topic of REST, not me. Given that you have this opinion, what would be from your point of view some criteria to decide which approach to use, when is which approach applicable?

That's a good question. If the problem that you are trying to solve is process-centric if it's a business process you are trying to implement, you can implement it with REST but I think there is a more natural mapping to web services and operations at that point. Versus if the problem is more data-centric then it's absolutely natural to map it to REST in that case. It's an art really yet, and I don't think anyone is in the position to give some criteria saying "Here is the rules to run through and you will come up with the right answer". If enterprise architecture was that simple, we would all be jobless. So luckily it is not because we have something to work on. But it's really a tough problem, there is a lot of technology alternatives out there, and it's not only these two, objects haven't gone anywhere, those have their scenario and it still makes sense to build a CORBA system possibly. I am not a fanatic on either end of it and my general way of thinking is there are so many different technology alternatives and you have to figure out the right one, I feel bad for people who are architects in companies who try to figure out what technology to build, because there's so much stuff and hype and lies, and so much of marketing garbage out there that is very hard to figure out what the technology is that works right and what is nonsense and so forth.

5. You mentioned that it's about 10 specs. Could you try and enumerate them?

The most important one from a service oriented perspective, is the thing that is used to describe what the service does. WSDL is the one that people are using for that. There is a whole diversion of the spec that widely deployed, WSDL 1.1. But WSDL 2.0 is a hugely improved spec; it's really not even a WSDL, it's a completely different language in many, many ways. Unfortunately adoption has been slow yet, it just came up this year, and it is going to take some time to major vendors have to got to revisions to get to that point, if they want to get there they have to have a much better way to describe services. The other key spec is of course is the base wire protocol that everybody uses which is SOAP, and there is a series of specs that extend SOAP with security, with reliability, transactions, and so there is WS-Security, WS-SecurePolicy and WS-Secure Conversation and WS-Trust, those are the four security specs that matter. On reliable messaging there is something called WS-ReliableMessaging, in transactions there is something called the WS-AtomicTransaction, WS-BusinessAgreement and WS-Coordination. There are many applications where you never touched reliable messaging or transactions. So a large percentage of the people who built web services would never actually end up invoking these things, and in fact most people who actually use web services shouldn't be knowing about these specs. This is the underneath infrastructure that people like me who build web services should know about.

6. Do you expect WSDL 2.0 will be adopted soon?

That's a difficult question to know what the right answer is. We are doing everything we can to help adopt it by making sure in all the software we write, if you put a ?wsdl2 you get a WSDL 2 document. We have done that for JavaScript, for Java, C, PHP and so forth. It is going to take some time yet. The reality is that the way web services got adopted is by having the major players adopt it and say "This is what we are doing" and that drives the industry. And that hasn't happened yet for WSDL 2 because it shipped after Microsoft WCF shipped and a bunch of the other things shipped. And now we have to wait for the next major upgrade before people will seriously consider doing a significant change like that.

7. Can you tell us a little bit about what kind of products your company builds and the way it builds them?

WSO2 is a little bit of an unusual start-up company. From day one when we started with the idea that the entire enterprise middleware platform is garbage out there right now and we wanted to rewrite it with the following mind set basically. Web services were created as a mechanism to make rich interoperation between companies. I am going to take an analogy, let's go back to the beginning of the Internet. IP is called Internet protocol, because back in those days I was on my own network, you were on your own network, I might be running Chaosnet and you might be running SNA, once maybe you sent messages out you dropped down into this ugly little thing called IP, send a message, and go on with our lives after that. Web services are treated the same way today: I'm running on my Java system, or my CORBA system, or my .NET system, you are on something else and we drop down to web services when we need to communicate.

Of course what happened over time was the need to communicate once in a way becomes more and more frequent, and what's outside becomes inside, there is no boundary saying "He's outside I'm inside and we speak different languages". Now we have IP in hardware basically and there is no other network anymore, there is no other network protocol, everything is gone. Same analogy with English: English is a language of business of the world, and if you really want to play business in the world you can't be sending the external message in English converting it into Singhalese or Tamil or Russian or whatever, because then you are always catching second fiddle because you are not really playing the language. That's the way we thought of web services and say "Look, web services is designed as an interop protocol, but if it is good enough to really do interop, it's really good enough for me to use as an implementation mechanism".

Our mindset is to say, "What is the new kind of mid sector that you need to create a set of products which make future looking applications which are always talking to other people, always are operating over the web, and what are those products?" And that is what we are trying to figure out. When we started two and a half years ago WS-* was the right answer for all that, all the things we started building was based on that. We have an application server we built on that, which is a way in which you can take some logic that you have and make it into a service. Over time obviously the WS-* world has evolved to be a little bit more tolerant of REST and lighter way of purchase. In fact in our platform if you write a service in Java or JavaScript, you can automatically get both a web services interface as well as a REST interface. I shouldn't say a REST interface, it's a bit risky I should say a native HTTP interface, Sometimes RESTy, sometimes not. The idea is with application server product, you have logic, you want to make it into a service, use that product. And we have a bunch of other products that we are building as well.

8. When you say that it's an application server, is it based on J2EE, or Java EE?

Actually no, that's another major decision we made. Java EE was a set of technologies that were built in order to make "portability" work basically. A technology built at a time when intranet integration was the problem that had to be solved. And we don't find that appealing at all. What we basically say is: "I don't need a container framework that works like that". What people need is basically write a POJO in the case of Java. And then we take care of doing everything else. So it is not a Java EE application server, it's a web services application server.

9. What other products do you have?

We also have an ESB, which again is not like the typically when you say "ESB" you think of heavy, big, ugly, slow, complicated and so forth. With our ESB, we basically started with the mindset that the application server is what you use to make an endpoint to host some logic. What we wanted as an ESB was something that I could stick my nose in the middle between A and B talking to each other. It's really a mediation system, I can intercept messages, I can route them, transform them and so forth. In fact one of the ways in which you will be able to deploy out ESB is actually as an HTTP proxy server. It's simply a part of the infrastructure and you can embed rules ... it's really like a firewall model that we have taken, unlike most of the other ESB vendors who have much more of a programming environment model.

In our case you write some rules and say "Hey if I see a message from Stefan to Microsoft block that or log that or whatever" that kind of stuff. I can write some rules and then of course we do allow you to write some Java code and execute that too, but our expectation is that most of the people will be able to write these rules and that's all they want to do. The third product that we have started building is an interesting one called the mash-up server. I am one of the people who created BPEL, which is another spec that a lot of the WS-* haters love. BPEL basically is a workflow system, it is a very good workflow system, it achieved what we were trying to achieve for the workflow space. But it didn't achieve anything for the service composition space. The reason is it's complicated, it's complicated for a reason, and it's a necessary thing for the problem space we were trying to solve with workflow.

But if what I want to do is take your service and take somebody else's service, mash it up or compose them and create a new service, BPEL is major overkill, a lot of XML junk you have to write, you get to write WSDL, Schema, it's a really complicated language. So what we set out to do is say ... If I am a JavaScript programmer, people are happy doing mash-ups on the Web right now, they write JavaScript code that invokes service one, service two, mashes it up and shows its UI on to the user. The only problem I see with this model today it's that it is not recursive, that it is I can't take your mash-up which mashes up some data that you have access to, and mash-up your mash-up, basically. That's what our mash-up server does, it's essentially creating JavaScript environment so that I can take a bunch of mash-ups services out there, mash it up, create a new service. And yes I can stick on an HTML full Ajax UI on top of that, I can use it as an application but you can come in and invoke the mashed up service from me. And now we are recursive, the idea being that if you build that ecosystem up then you are going to build a lot more services that people can use.

10. What is the relation of your products to the Apache open source stuff?

My personal relation to Apache goes back to 1998, I contributed to the scripting language extensions for Xalan, which allow you to invoke Java code and external code basically. After I moved back to Sri Lanka I started an open source foundation in which we started doing various Apache projects, that's why most of the Apache web service projects have these funny names because they were done by mostly my students working on various projects. So I am a long time Apache person. The way we basically do everything in WSO2 is all of the core components we build as Apache projects, and the only thing we do in WSO2 is package them all up and make it easier to consume and then give them away completely under the same license. All the integrated stuff we release under the Apache license, in fact the development is done completely openly and we try to set the same kind of processes that Apache uses for our own development. So you can come to our mailing list and see the arguments we have, what bugs we are working on, what features are going to come up in the next versions, when is the next release, you can see the issues, everything basically. We have a very close relationship with Apache on 95% of our developers we have all Apache committers, and those who are not are working to getting there because almost everybody touches 1 or 2 or 3 or 4 or 5 Apache projects. And then we package it all together and offer a value-added basis, slightly value-added version. We also send support for the basic Apache components, we don't have a problem with that and we are perfectly fine if you want to use Apache Synapse or Axis 2 or whatever it is that you prefer to use.

11. The application server is basically based on Axis2?

The application server is basically Axis2 integrated with security and RM stuff and with an Ajax thrown in that drives a bunch of services that control the runtime.

12. ESB is Apache Synapse?

Yes, ESB is Apache synapse plus again visual controls built into it. The mash-up server we haven't done in Apache, we ended up doing it in WSO2 as an open source project. The reason is it is not really a web service project that would belong in the WS project for Apache. It is interesting that all the work we done in the mash-up server, really doing it and really deep, binding of web services and REST into JavaScript. The product model is quite cool actually, you just write a JavaScript function, and just annotate the function with some stuff. The way you annotate in JavaScript is not like Java you don't put at sign and a lot of nonsense, you simply add properties to the function object and it is a very lightweight, very easy programming model. And then you drop the JavaScript file into a directory and it immediately shows up you got an URL for it and you browse for it and you get a UI for it, and if you say "?stub" it will generate you a JavaScript stub for it. It will be able to invoke that function as a service. It's a different kind of product, we didn't find, didn't feel that was the right community in the world that we were living in within Apache, and we really didn't want to go through the whole incubation process because we don't need to, we are a bunch of Apache people and we decided not to do it there.

13. The last product we talked about earlier was the registry thing. Do you want to tell us a little bit about that?

Sure. It is a registry and a repository and it has absolutely nothing to do with UDDI. It's essentially a writeable website that we are building, so it's like SVN plus Atom plus REST. In fact it has nothing to do with web services at all it's not even using Axis2. It's starting basically from scratch again and we said "What do I want if I am putting up a registry today not using the 90s technology but today what I really want is something like a Wiki which is sufficiently access controlled for an enterprise setting, that means hierarchical security management and we wanted it to be a little bit more structured, if I'm in an enterprise scenario I am not going to put up a Wiki and say "Guys put up your stuff here ...", that's not good enough.

So you need a little bit more structure, you definitely need versioning and you need to be able to get feeds for everything, and you should be able to have some level of intrinsic built-in knowledge about certain things. For example if I upload a WSDL document to the registry and if the WSDL imports a Schema, the registry should be able to know that there is an external reference and should ask the guy if you want me to suck that in, too, so that I can actually have a completely closed world within the registry. That's the kind of thing we are building, it's interesting, it's sort of a writable website basically, using AtomPub as the way to implement the actual writing into the website, it's completely resource based, everything, and it also has a lot of community features, if you create a service and stick it into the registry, you can now allow anybody who is one of your friends to comment on it, and apply some tags to it and so forth. It has all the web 2.0 cool features, yet it is a full scale enterprise registry, but it is going to be very lightweight, it's a little bit of Java code, you can write that as a .war file, and you we'll also have clients for it, you'll have the HTTP protocol for it and then implementing some clients, it will also have a Java client for it, we are working on a PHP client and a JavaScript client as well as Ruby and Perl and a bunch of other languages, and C.

14. Speaking of scripting languages or dynamic languages, do you want to tell us a little bit about the integration for Web services that you are doing?

Yes, I have a long history with scripting languages; I am the one that did something called bean scripting framework which became now JSR 223, which is basically the way in which to integrate scripting languages into Java. I have been a long believer in scripting languages as a solid programming model not just a hack job that you do on a side. When we started WSO2, about 2 and a half years ago, we believed very strongly that Java is not going to be the "winner". Today if you are building enterprise middleware, you have to have a Java story otherwise nobody is going to take you seriously. But I don't believe for one second that Java is going to keep on growing.

I think Java is a fantastic language, a great environment, and it has a solid audience and it is going to have their audience, the audience is not going anywhere, but I don't see it growing and becoming 60% of the development community or anything like that. What I see is there being a myriad of languages; people are going to be programming more and more in more than one language. When we started we decided we want to implement this functionality and make it available for a bunch of languages. How do you do that? One choice is you implement in Java, PHP, Perl and so forth. But there is a better choice which is implement it in Java and in C. If you implement it in C everyone of this scripting languages typically is written either on top of the JVM or on top of C. So with the C stack we have integrated it now to PHP, to Perl, and just released the alpha for the Ruby release, and next it will be Python and probably Erlang at some point, because all of these things are written in C. And also even the Java version, that's really why Axis 2 went so forth to not implement the JSRs natively.

If you look at the JSRs, they only apply to Java the language, they don't apply to Groovy, they don't apply to JRuby, Jython, they don't apply to all these other languages that sit on top of the JVM. So if you go and make a web service stack which is all about JSRs it has no applicability if you are in Groovy. We created a core library, and now we are also working on doing Java bindings to other languages, we are doing Groovy binding, JRuby binding, in fact we are trying to keep the JRuby binding and the Ruby binding exactly the same. So you could take the same Ruby code to invokes web services and run it either on a Ruby on Rails or Ruby or on a JRuby.

15. Using this integration, can I only invoke web services that are running on another platform? Or can I also build my own web services?

Yes you can do both directions. The PHP code that we have done is for both client and server, the Perl code we only did the client so far we need to finish it, but we felt there was more interest into the Ruby stuff so we switched into doing Ruby right now, the Ruby client code is what we released, we are working on the server side code and we are also working on the JRuby server side. In fact for our mash-up server, we are taking all of the mash server functionalities and enabling them for JRuby. At some point we will have a mash-up server JavaScript edition and a JRuby edition. In fact it might be the same, and if you drop in a .rb file it works in one way, if you drop in a .js file it works in another way.

16. Switching topics a little, your company, or most of your company, is based in Sri Lanka.

Very much. I spent 16 years in the US and went to school in the US and then taught at Purdue University for a while and worked at IBM research for four years in New York, and then we wanted to move back. About 1999 we decided to move back, in 2001 we actually went back and then I continued to work for IBM research from Sri Lanka, telecommuting basically, and after 4 years I decided to start this company. By then I had already started this open source foundation, we had a bunch of open source work. But Sri Lanka is actually now considered the largest contributor to open source for all of Asia. 5% of Apache committers are now from Sri Lanka, not all of them are from WSO2, there are other people doing various things and we have a bunch of open source activities. I have a lot of passion in trying to bring open source development, not usage. I don't really care whether they're using X or use Windows, that's not my interest. My interest is in saying open source community is a bunch of technical experts all over the world, and nobody in Asia is contributing. And I don't care about the other countries as I care about Sri Lanka, so I want people from Sri Lanka to contribute. Because I lived in Sri Lanka it was natural to set the company up there. And of course you get the benefits of the cost advantages, the global structure, but we are a truly global company, we have people in UK, Boston, California, and we do everything online, we live on mailing lists, get a crazy amount of emails, IM, phone... it's a distributed structure.

17. What do you think the future has in store for integration from middleware, what is your expectation of the Next Big Thing?

The next big thing. I think I am going to give you a boring answer, I think there is not going to be a Bext Big Thing, I think it is going to be a bunch of little things, it's like in the language world. There was a time when everybody was trying to figure out what was the next big language. There is not going to be a next big language, there is going to be a lot of them. The same thing is going to happen with integration. I don't think REST is going to kill web services, I don't think web services is going to kill REST, I don't think CORBA is going to kill web services, I think we are going to have more and more variations that are more specialized to specific scenarios. And the other big thing that will definitely happen is figuring out how the Internet plays into things like middleware. I don't thin we know yet how hosted and online approaches for things like middleware will ship out. One of the things we are trying to figure out is what kind of stuff that we built should be hosted, how should it be hosted, how should people be able to write a mediation, or a mash-up, hosted mash-up is easy because you have web services, that's natural. But also you have to think about how does hosting work both in the public Internet scenario as well as in a corporate scenario.

18. You are referring to models like Amazon's S3?

I am actually extremely pressed and interested, we are thinking deeply into trying to figure out how to use S3, EC2 kind of model to build hosted infrastructure. We are doing some work now to allow easy deployment of our products on to EC2; we'll try to get to a point where we will have my image that you can download from our site, but going further what you really want to do is to combine EC2, S3 and SQS and say "Look if you want to deploy mash-up server let's say, deploy it to EC2, all you do is type in your account information and it will automatically scale out, because using SQS and so forth it is easy to scale out. That's a level of hosting that now a middleware vendor can provide to a customer and we don't know how it is going to shape out yet, we are going to keep playing, and do all kind of stuff most of which would probably not be right, and keep integrating till we get it right.