Interview: Pete Lacey Criticizes Web Services

"InfoQ: If I understand you correctly, you?re referring to scaling to Internet dimensions. But not everybody needs that, do they?

PL: It?s true that the WSF prohibits scaling to this level, but, no, that?s not what I?m talking about."

Any data on how WSF "prohibits" scaling to that level, I've seen an awful lot of bland claims like this recently but without the actual data to back it up. A pointer to the many massively successful REST commercial implementations and the many massive failures of WS would help.

Now personally I don't care two hoots which one wins, we just need _one_ that everyone agrees on, and I agree that SOA works no matter what the IT infrastructure is, but if REST is to actually win it would be nice to do it based on facts and examples rather than rhetoric.

We don't just need _one_ that everyone agrees on. That's the reason we have these on going debates. Different problems, different solutions. REST isn't perfect for all problems, neither is SOAP.

I'd be more curious to see how many true WS large scale successes there are. People seem to be using the subset that lets them get their jobs done. Claiming a small subset of WS-* adoption as an overall WS win is a bit of hype as well.

Steve,

I know you won't find this argument legitimate, but here goes. Since REST is based on the exact same principles as the Web, and the Web has proven beyond all doubt to be Internet scalable, therefore REST must be Internet scalable. In contrast, since SOAP web services are modeled after distributed object systems, an architectural style that has proven to not scale beyond the enterprise (and I'm being generous), therefore SOAP web services can't scale to Internet dimensions. QED.

I admit that we are not tripping over RESTful web services, but then neither are we tripping over SOAP web services. I think the pessimism I and others exhibit around SOAP is based on how little progress has been made with SOAP WS despite the millions of dollars and thousands of man-hours invested. It's time for the other guy to have a shot. I'm betting he succeeds.

This is a great argument Peter. Unfortunately its completely bogus. Most websites are not built according to REST principles. They scale well because many engineers work very hard to make them scale. And also you are looking at the websites that do scale. There are plenty that don't. That doesn't mean that REST doesn't scale.

Similarly there are plenty of SOAP services that scale. For example, I believe that eBay processes 40 million SOAP requests a day. I'd call that scaling. Similarly Yahoo Mail is based on SOAP calls and must be handling a similar number.

Finally, its wrong to sayt hat SOAP is modeled after distributed object systems. SOAP may have been designed as a replacement for those, but the SOAP1.2 and WS-I profile expunged all of those features. SOAP is just an XML messaging model and scales as much (or as little) as any messaging model does.

Paul,

I was very careful to make the distinction between transactional scaling and process scaling. I have absolutely no doubt that one can build SOAP-based systems that can process all the messages it needs to. Where SOAP web services fail to scale is along most of the other dimensions: developer productivity, operations, deployability, etc. In the context of Internet scaling I will add evolvability and reach. REST styled services are inherently available to anyone, they can evolve with minimal impact to the client, and they can be fully integrated into the Web as we know it. When I say that SOAP web services can't scale to the size of the Web, that is what I mean.

It may be true that SOAP has evolved on paper to being just another XML messaging system, but so long as the tooling (IDEs, platforms, intermediaries) continues to generate proxy objects, treat XML as a serialization mechanism, and pretend that the network doesn't exist, SOAP is still walking and talking like a distributed object system.

+1, what has changed architecturely is the emergence of the "request" over the "method". Distributed systems are based on the premise of providing distributed access to "an object".

Web systems are built to process requests. The "thing" that processes request is a sometimes called/identified as a service.

More generally Peter, I think your entire argumentation is old and belong to this strange category of behavior of the software industry that I call "generalism". I have often noticed (and occasionally had to fight) that software engineers generalize way to easily the occurence of a particular pattern or concept. I don't know why. Physics from Newton to Einstein should have made it clear that generalization only applies to a "context", there is hardly any universal, except the arbitrary.

The question that is IMHO important what problem are we trying to solve (agree on that) and then decide what solution is best adapted to solve a particular problem without suffering from "generalism".

SOA is (painfully) creating a new application model where "distribution" is the norm, or rather "connectivity" is the norm rather than monolithism. Connectivity is different from distribution. The fact and the mater is today you can't build a system in isolation of its environment. The question is how you can do that? Building a monolithic system we know how to do, building a monolithic system that scales we also know how to do. Building connected systems is different. Sometimes, REST is just fine and applying a REST solution to a REST problem is a great architectural decision. Now can REST be used to build composite information systems? yes? good luck.

Now you can argue that the complexity (or poor design) of the WS-* stack is neither applicable to building this kind of systems, maybe. Having my name on a few standards does not bind me to a community that has most often discredited itself by its "individualism" than shined by stellar designs. But thinking that connected systems is a "simple" problem is naive and misleading. I would recommend that when you promote a solution, you also promote the context in which it is applicable. Of course, that is the hardest of both exercize. Ever hammered a screw?

JJ-

Peter,

I enjoyed reading your criticisms of WSDL/XSD. I respectfully disagree with some of your conclusions. I work for a company called Mindreef, a Web services testing company, and co-authored our dynamic invoke forms. Clearly, such tooling benefits enormously from the WSDL/XSD...up to the point of types on the elements. Enums and booleans aside, the STRUCTURE of the message is far more important than writing greatly nuanced restrictions on element content. On this point, we agree.

I also agree that when authoring a Web service, manually processing the XML is the way to go, as it is more maintainable, and far more amenable to processing the constructs schema can specify, and being adapted to changing contracts over time

Taking the time to author the XSD gives testers and developers a contract to rally around, and enables tooling such as ours, enabling more rapid development, testing, prototyping, better human-documentation, and so on. Further, I believe toolkit generated clients are 'OK' (though should be re-architected in such a ways as to be robust in the face of changes, but we are uncovering these best-practices via these growing pains. Subtyping is nothing new, just being rediscovered. I agree that these problems do in fact come from the mismatch of XML documents to 'Objects')

Finally, XSD validation is useful for both developers, testers, and support staff to quickly highlight some of the most common interoperability problems, ensure corporate compliance, or compliance to XSD defined industry standards, etc...

Just because WSDL and XSD are gnarly, why throw the baby out with the bath-water?

- Aaron

Jean-Jacques,

I think your entire argumentation is old...

Paul, I'll call your argument completeley bogus as well, just to be fair :-) There are likely more RESTful Web services than those based on SOAP 1.2, since most WS stacks still prefer SOAP 1.1. AFAIK, the WS-I Basic Profile 1.1 (and even 1.2? not sure) still embrace SOAP 1.1.

The Web is RESTful to a significant degree because it's a) largely read operations and b) uses lots and lots and lots of URIs (e.g. every book on Amazon has its own URI). True, the Web doesn't use PUT and DELETE much in practice, but I strongly estimate URIs are used RESTfully with POST and GET semantics about 1000 times more than they are being used as SOAP endpoints (on the Web, that is).

Aaron,

First, though I haven't used it in a while, I love SOAPscope. Good work.

From your comment, I think you're saying that just because WSDL and XSD are abused doesn't mean we shouldn't use them. If so, and in the context of SOAP services only, I agree. Sure, if there were a "better" WSDL, I would be happier. I can certainly see how tooling benefits from a service description.

Regarding message definitions, I think we're on the same page. As I said in the interview, there's a benefit to having a rigorous description of the information passed between systems. I just don't think XML Schema is up to the job, far better would be Relax or Schematron. And I also think that today's tools are making more brittle systems than necessary. But WSDL and XSD are the devils we're stuck with, so, if your developing SOAP services, try and keep your tooling in check and make the best of it. Shame, though.

They are definitely the devils we are stuck with, and it is a shame. I could liken it to the damage one can do in C if abused, and personally I wish were all coding in ML! But I do expect the WSDL/XSD adoption to go through some similar growing pains as the early web did w/ HTML/HTTP. Do we really use all the HTTP verbs , and for the ones we do use, do we use them in the way they were intended? We've 'evolved' into some simple common practices, and the web is humming along nicely. WSDL/XSD will go the same route.... Committee-ware doesn't bear fruit easy, but that might be the price of interoperability.

1. The WS-I Basic Profile 2.0 will address SOAP 1.2 . From our experience nobody but Microsoft projects using WCF to WCF.

2. I've never met anyone who has implemented a real REST style service. Most web sites I've seen use a handful of urls often of evolving and gnarly format, tons of server state in session and DB, adding over plenty of poorly documented url params that really dictate the resource in question. I'm not knocking REST I've just never seen it done in the architectural purity that you would be led to belive.

Jim

My favorite example (currently at least) is the Blinksale API.

Peter:

if we agree, I would recommend that "put to rest" this now antiquated and rather stupid debate of "soap" vs "rest". It would be far more efficient to focus everybody's attention where each stack works best and what are the usage patterns for each technology in their own context.

The WS-* stack was developed in the most cahotic fashion. It deserved everybody's interest including the vendors and by far the interest of the customers/users. BPEL 2.0 is not out yet, how smart is that? who wins? WS-TX went through 3 iterations since 2001, who wins? Now the WS-* is what it is, could it be simpler? maybe, is it that bad? not to the point where it is not usable. SCA fixed one of the major issue in the stack.

Now, I think that the problem that you are really talking about is MDD. Is MDD the right approach in general or not? What the WS-* stack underlying metadata approach ignores is that it is MDD, and thus is creating a wave of metadata that needs to be managed to be effective (tools, repository, versioning, ...) of course you have to complain about how painful and complex this looks like. Again, anybody entering the SOA world without a metadata management strategy compatible with their infrastructure of choice would probably rapidly face a wall. That has nothing to do with the architecture of WS-* itself.

I have written several articles on the "SOA" stack, not just WS-* if you care to read one of them.

www.sap.info/index.php4?ACTION=noframe&url=...

you might appriciate the order behind the chaos.

JJ-

www.sap.info/index.php4?ACTION=noframe&url=...

Pete, I am tripping over SOAP web services. In the last two weeks alone I've been advising 3 different companies on how they work with external entities, in not one case did I suggest the use of WS-*, but in every case they had already selected it as it was the only solution that all parties agreed upon. Those 3 companies I dealt with are in total dealing with 15 external parties.

Saying that "its time for the other guys to have a shot" is classic IT ADD, WS has in terms of "prime time" been around for only 3 years now (I know when I did WS in 2001 I was a renegade maverick against the EAI crowd), and its completely bizarre to claim that in three years it hasn't progressed enough.

How about for a change IT doesn't go "oooh shiny string" and actually concentrates on getting ONE solution we all agree on for what is easily the least important part of a project these days, namely the actual shifting of XML docs from A to B to achieve a business goal. Its the goal that is important and the XML shifting that should be commoditised. It is not time to give the other guy a chance, its time for us to start focusing on real business value rather than our views of technical excellence.

And you are right :) I don't (as someone who has built quite a few commercial websites) consider "the Web" to be a REST reference :) I would say (and its at least true in the systems I've architected) that modelling WS systems around distributed objects is a bad move, you should be modelling them around distributed services and a service != an object.

"<Movie>
<Title>Star Wars</Title>
<ReleaseDate> 1977</ReleaseDate>
</Movie>

Why should it choke if it gets
<Movie>
<Title>Star Wars</Title>
<ReleaseDate> 1977</ReleaseDate>
<Runtime>121</Runtime>
</Movie>"

I'm confused if the Runtime field is optional and using an XML parser which marshalls/unmarshalls only the fields required... why would this break WSDL based services? If the XSD has changed to include the new non-mandatory field then the schema validation will pass. Your argument appears to be that XML parsers can't handle the addition of non-mandatory fields rather than an issue with WS-*.

That is a pretty easy problem to fix and I know that some of the XML marshalling stuff I've written back in the days using SAX and XPath would have handled this fine. And this is before you put mediation in, which makes even more complex changes solvable.

Of course it the field is mandatory and important then of course it should choke as a client that doesn't process that field might be doing something very very bad (e.g. if the extra field was "will get arrested if you watch this")

Not an issue... please move on.

I think it is an issue because it represents an additional layer to maintain. It may not cause the ground to shake (like the 1000's of specs in that poster ;)), but it represents a pro on one side and a con on the other.

"If the XSD has changed to include the new non-mandatory field then the schema validation will pass"

What about software that hasn't yet been updated to this new schema? See;

www.coactus.com/blog/2006/12/validation-conside...

First, this is not a problem with SOAP per se, but a problem with the tooling. Many of the issues I raised can be avoided by keeping the tools in check. It's more work for the developer, but it pays off in the end.

The reason this change to the message tends to break client code is that the client was likely built from a schema that didn't include the <Runtime> element. But now the service and the schema have been updated to include this field, it's even been made optional. But the client hasn't been rebuilt. So when it makes its getMovieInfo() request it gets back 3 elements when the serialization code and the generated proxy object were only expecting two. Result: blam! That's why I say that code generation and serialization techniques make clients and services too tightly coupled. Again, not a fault of SOAP, but the way it's commonly practiced.

Doesn't that depend on how the schema association is done? If the generation is 100% static then there is an issue, if however the validation is done based on the schema referenced in the XML document then there will not be an issue.

Sure if the XML doc has changed so the structure doesn't match then there is an issue, but that is a different use case.

And I think we are going to agree to disagree on the validation point validation up front works for me.

So your critisism is of the way that XML marshalling tools are built and with the short-cuts that tools vendors take. I'm not going to take issue with that, but it would be good to make that point and calling the tools vendors to account rather than throwing the baby out with the bath water.

While I do think that you have a valid point here, please refrain from spreading the nonsense of equating SOA with SOAP/Web Services. SOA is an architectural paradigm not tied to any specific technology. And that's where it value lies. Implicitly assuming that in order to implement an SOA you need to use SOAP/WS-* only serves to cloud its true nature and thus is actually dangerous.

Olaf

Steve, I claim all current WS stacks break with the XML example Pete gave. Schema validation is practically never done at runtime (it's usually deemed way too expensive), and 100% code generation is the norm. Sadly.

Olaf, while I understand your point, please don't accuse someone of actively "spreading nonsense". I'm not aware of any publicly available and accepted SOA definition that shows you are "right", only that there are many who share your view, as well as many who don't.

As to my personal opinion, I have seen large numbers of people who agree with you that SOA is an abstract style, and end up doing WS-* only anyway. YMMV.

Agreed, the tone of my posting may have been a little harsh. Still, I maintain and will continue to do so that either explicitly or implicitly assuming that an SOA requires the adoption of SOAP/WS-* is not only simply not true but actually dangerous. Given the inherent complexity of SOA as an architectural style any such assumption that keeps you from choosing the technology that best fits your specific context may well cause your project to fail.

There is another, more general gripe I have with this equation: Prematurely choosing a specific implementation technology introduces artificial mental barriers which tend to keep you from concentrating on your specific problem domain and its unique qualities. Yet being able to "choose the right tool for the job at hand" is IMHO all the more important in an SOA, since these systems tend to be so complex.

Olaf, I understand your point. In my personal experience, though, technology decisions are often postponed far too long -- leading to more and meaningless discussions because they remain at a level that is way too high to be useful. I think we can agree that prematurely choosing a specific technology approach is not a good idea. I'd just like to add that to choose it too late is just as bad :-)

Agreed, and I share your experiences -;

Cheers

I'm not sure they all do, I'll have to go an check now :( But anyway the cost of validation is (IMO) pretty minimal (I remember doing Ada & Eiffel in the early 90s which did all this and more and it was okay. So the issue is the XML bindings as implemented (its not the WS stacks, its the generated code, hence the reason its the XML bindings). Picking this implementation issue as a criticism of WS-* when REST has so many holes (MIME types anyone?) is more than a bit rich.

Lets pick on the vendors (and people like Paul Fremantle :)) and the XML binding technologies, rather than the current implementation sets. This isn't exactly a hard case to solve... its probably just that they haven't had the use cases and real world scenarios to demand it...

Which sort of makes you think....

SOAP/WSDL and XML binding technology are different things. Whether to use an XML binding technology is an implementation decision.

- I can create WSDL/SOAP services that use them (e.g. JAX-B) or not (e.g. BPEL, DOM)
- I can create REST services that use them (e.g. JAX-B again) or not (e.g STaX/SAX/DOM).

Issues such as the requirement to support runtime validation or how you intend to support service versioning (both at the schema, semantic and implementation levels) may influence your decision on whether to use an XML binding technology or not. SOAP/WSDL currently doesn't push you one way or another.

This is really an argument against the tools, not the specifications. Many have said this but I'll reiterate - do not throw the baby out with the bathwater. If the implementations of the specs are bad, let's fix them. In that context these debates are very valuable.

However once someone starts suggesting that we just throw it all away and start over again that isn't very constructive, since Web services do represent significant industry progress.