unREST as the new REST?

I remember when I defended my thesis a few years ago - Theory and Practice of Web Services - it made quite a stir in the committee that I announced that in fact, SOAP is not used mostly, but nor is "faithful" REST. I tried to bring the two together, syntethizing the good parts of both.

Now it seems it starts to happen.

Don't be shy, forget the 4 little HTTP verbs, you can even define your own: Query/Do/Notify/Synchronize is a perfectly good one.

Yes, exactly, we threw away a lot of the advances that SOAP/WSDL brought, such as technology neutral, machine readable contracts, a true forwards versioning capability, bi-directional interfaces, asynchronous message exchanges, when we could have simply took the best of both world and move forward. Instead we went back to the 80s.

Slava:

I don't understand your point. It is clear that the REST "uniform" interface is broken. You are forced to use POST for some queries, people use DELETE artistically, they can't figure out if they should use a PUT or POST.

No, "unREST" recognized the important of a uniform interface. We had several in the past:
a) SOAP's uniform interface was SEND / RECEIVE
b) RDBMs used CRUD for over 30 years
c) in OASIS ebBP we defined one too
...

All I am saying that "one size" fits all doesn't make much sense, different domains need to think differently about the classification of message exchanges. A Query/Do/Notify/Synchronize is far more representative of the interactions between systems than GET/PUT/POST/DELETE.

A uniform interface help you build specific support in your infrastructure for each time. Just like HTTP does.

>> I also believe, message specification can be applied to REST
I am glad we agree, because, they are actually much more human readable than REST's Human Readable Message Descriptions.

Thanks for the follow-up, Jean-Jacques. My point was that REST and unREST are just looking at the API from different abstraction levels. I agree, GET/PUT/POST/DELETE are not at all representative of the interactions and Query/Do/Notify/Synchronize are. But while unREST (as I can see it) gives human-readable names to domain-specific interactions, I'll take the liberty and state that REST defines domain via the resources and generic actions that can be performed on them, where data (or attributes) sent in RESTful requests are regarded as providing the details of the interaction. So unRESTful verbs can be regarded as "human-readable domain specific alternative" to the RESTful "resource + HTTP verb + (optional) attribute(s)" combo. But... for unREST, all methods are born equal - destroying a resource is just as "safe" as querying it, updating is not different from creating. Is this where you suggest to take "configuration over convention" path by specifying messages and using contracts? REST takes "convention over configuration" path here and provides means to categorize the interactions - via HTTP verbs (maybe not as fine-grained as one could achieve with a specification, but IMHO, sufficient enough). If some people are arbitrary using DELETE, PUT or POST (or whatever) HTTP methods, it doesn't mean REST fails as an approach. I'm rather inclined to think, they failed to grasp the basics of REST. To put it simply, if you want to update a resource - use PUT, you want to create a resource - use POST, delete a resource? - use DELETE, and all querying is to be done via GET. I'm aware of using POST for querying, because of HTTP limitations on the length of the query string, but it's not the problem of REST, is it?

Oldie but Goodie:

wanderingbarque.com/nonintersecting/2006/11/15/...

Developer: So, my boss was playing golf this weekend, and now I have to ‘quote, unquote’ SOAP-enable the enterprise, but I don’t know what SOAP is. Can you help, SOAP Guy?

SOAP Guy: Sure thing. First, SOAP stands for Simple Object Access Protocol.

Dev: So it’s simple?

SG: Simple as Sunday, my friend.

... and then the fun starts. :-)

Slava,

really, all I am suggesting is that you choose the uniform interface that's most suited to your needs. I am not suggesting that one is better than the other. When 80% of your interactions are simple queries, maybe REST's uniform interface works. If you need some level of bi-directionality or deal with intermittent connectivity like it is often the case in mobile technologies (Push Notification), REST's uniform interface doesn't help at all. It is actually quite a broken there.

>> generic actions
There are no such things as generic actions. All actions are specific for very good reasons. Translating a specific action into a generic VERB and a specific noun, does not make it any more generic.

>> safe
Safe is a property of the uniform interface. In the Q/D/N/S uniform interface there are 2 safe methods: Query and Notification. Actions are intrinsic to any "inter-actions". Please, note that QDNS is just an example of uniform interface. I am not recommending to replace one by the other, I am just recommending to make a conscious choice of the uniform interface based on your domain.

>> for unREST, all methods are born equal - destroying a resource is just as "safe" as querying it, >> updating is not different from creating
I am not sure I made such claims. Why would you think that "Updating" or "Creating" are valid interactions for any domain? Are you talking about the uniform interface level, or the inter-action level? Who would ever want a client to decide where they should update or create something. That's an obvious coupling that even Roy is pointing out as a major mistake people do when they claim to be RESTful. You seem to be using the CRUD uniform interface reified on top of the REST uniform interface.

>> all querying is to be done via GET.
So what happens when the URL path is too long and you HAVE to use POST?

As I said, there is no superior uniform interface and all uniform interfaces can pretty much be reified on top of any uniform interface. That is not the problem. The problem is when you create hundreds of Web APIs, for a specific domain, say, mobile applications, the question becomes is the uniform interface you chose adequate? or say, each time you want to express a "notification" interaction you find yourself having to write a lot of documentation for explaining how it happens using a combination of interactions (polling, XMPP). There is the problem. How do you express an XMPP interaction in REST? Yet XMPP is extremely common in the mobile world, and mobile applications are taking over. So I am not sure why I should be forced to use completely suboptimal technology to serve a major software domain.

>>specifying messages and using contracts
Once you pick your uniform interface, there there is no question about machine readable contracts. Can you tell me how much we won or lost going from machine readable to human readable?

>> REST takes "convention over configuration"
Could you please provide any evidence that any convention is reusable from one Web API to the next. I see and deal with a lot of Web APIs and not one has a set of reusable convention. I can read a WSDL file in less than 2 minutes and have a really good idea of what the service does and how to interact with it. I can read a human readable documentation for two weeks and still miss 20% of what the author has tried to convey. I am not sure where the gain is.

>>I'm aware of using POST for querying, because of HTTP limitations on the length of the query string, but it's not the problem of REST, is it?
In the end nothing is a problem, the only problem is the time it takes to build something, and we have gone from an era where it would take about 2 minutes to consume a Web service to now where it takes anywhere between 2 days and 2 weeks. So maybe this is not a problem of REST, but that's a problem I deal with every day, for me, it doesn't make much difference.

Do you have a link to your thesis somewhere? I would like to read it and provide feedback.

I think it's the first time I see someone talking about a "domain specific uniform interface" - which I did in the rest-discuss mailing list in the realm of implementing REST in a multi-protocol way, and being bashed in the process...

Didn't read the full article yet, but it looks interesting...

>>I'm aware of using POST for querying, because of HTTP limitations on the length of the query string, but it's not the problem of REST, is it?
In the end nothing is a problem, the only problem is the time it takes to build something, and we have gone from an era where it would take about 2 minutes to consume a Web service to now where it takes anywhere between 2 days and 2 weeks. So maybe this is not a problem of REST, but that's a problem I deal with every day, for me, it doesn't make much difference.

So what happens when the URL path is too long and you HAVE to use POST?

Antonio,

Don't waste your time with rest-discuss. My recollection: It is filled with consultants trying to sell snake oil. Nothing is more irritating than some nerd yelling at you "I KNOW HOW TO BUILD WEB SERVICES, AND WHAT I AM ABOUT TO SAY HAS NOTHING AT ALL TO DO WITH REST BUT I FEEL LIKE POSTING IT ON A REST MAILING LIST." Some smart people have traveled through that list and some smart people still post there, but they are routinely shouted down by people who have no idea what they are talking about. If you could link me to your original rest-discuss comments, I'd be happy to comment.

John,

let's take an objective measure. Let's look at how many Web APIs are truly RESTful on the programmable Web directory? There are 3500 public APIs, not two are the same, it takes hours just to decipher the logic behind each API and I am not even talking about signing requests.

Is that what "REST" brings us? We have now 4 years behind us, lots of data points, we are past the discussion of POST vs GET or PUT. We can see what people do, how they do, and that is not RESTful.

Jean-Jacques,

Your complaint is now much more focused. I understand it better.

Some points:

1) REST is not Rapid Application Development (RAD) technology. Microsoft tries to build RAD toolsets, such as OData. RAD toolsets have very different objectives from REST technologies. OData also has very different objectives from REST. As I understand OData, the objective is to give total power to the client to write BI reports. That will indeed become more common, but it is probably not something that should be considered a "Web service" due to obvious Denial of Service attack potential.

2) All APIs require understanding their behavior, and most interesting APIs will have higher-order behavior. Reasoning about higher-order behavior is very difficult. There is no silver bullet. Most likely, you will need a combination of good techologies (languages, APIs, IDEs, etc.) in order to eventually make building Web services easier. This realization is nothing new. The UML Model-Driven Architecture weenies have been saying this since before SOAP even arrived on the scene. Nobody listens to their good ideas, though, because they hide behind gross pictures and tend to approach the problem very differently than conventional ways you see on blogs covering SOAP or REST. MDA is a "better RAD".

Basically, put this way, your complains are "not even wrong".

But I also think your complains were wrong, as highlighted by very good comments on your blog by folks like Cyril Rohr. But I am willing to let that pass, with the assumption that what was really nagging you were my 1) and 2) above, and you are just thinking out loud trying to articulate 1) and 2).

Yes, I think you boiled it down correctly. If I may / add to it:

a) the uniform interface is unnatural to use, most people get confused, specially when it comes to translate actions into POST + noun.
<somedetails>
Examples include:
Pay->POST Payment
Sumit Order -> POST Order
Cancel Payment ?? DELETE PAYMENT or POST Cancellation? (I know the answer but not everyone does)

If you look at Sun Cloud API you have the notion of attach/detach server from a cluster
Attach -> POST ??? (attachment?)
Detach -> POST ??? (dettachment?)
<rant>
this is insaine.
</rant>
</somedetails>

b) Every one needs RAD, unless I am google, Yahoo or Facebook, I don't really care about the scalability of the Web, I care a lot more 1) how much it costs to build an API and 2) really important, what is the developer adoption.

<somedetails>
With the explosion of mobile apps, the API providers that make it simple to consumer APIs win, the harder it is the more developers drop off and your services are less used

<disclaimer>
I actually develop such MDE tools at

Do you have a link to your thesis somewhere? I would like to read it and provide feedback.

SOAP is 30 pages, it's as simple as it gets; it's only that 30 pages is more than zero, and effectively that's what an average developer is able to understand - so we have these "as much as I've understand of HTTP" interfaces nowadays; at least OAuth starts to gain traction :)

For a), I don't think switching away from REST will help.

Understanding the answer your your Cancel Payment puzzle is critical to being a good enterprise systems programmer. It would make for a very good interview question, actually. Too many accounting systems I see are confused on this issue. Or, if it is not the accounting system, it is the billing system. It gets worse for products like SAP with customization everywhere, because then customization programmers can customize the product to the point where it no longer is being used as intended.

As an example, I know one client who completely misuses their Charge Master. To provide an audit log for their Charge Master, they export us a snapshot of their Charge Master each day and we tell them what changed. We then use this information to provide point-in-time Business Analytics for them.

It's mistakes like these that lead to civil action lawsuits due to billing errors. I won't name names. Everyone can name at least one civil action lawsuit where customers of widely used services were over-billed and nobody at the service company could figure out what was billed when and why.


I can't make bad programmers, good programmers, and can't make bad code, good code. The reports can be correct, but the workflow can be screwed up beyond belief. Higher-order behavior is very hard for programmers to reason about.

Haha. It is 2011. We live in an age with AppEngine and BigTable. Give me a plaintext version I can feed through Google Translate. If I don't understand anything, I will give up. If I don't understand one or two sentences, I will ask for clarification.

Send me a mail to aadaam at the free mail service of the creators of those two apps.

John,

the problem is that REST, absolutely does not help the "programmers", it confuses them at best. Everyone feels that he or she too can become a guru and decide to use a header for X, a media type for Y and provide its own interpretation of what HATEOAS. Most of all, it also makes the programmer a "spec author" who now has to create a human readable documentation of his or her choices.

As you know business is complicated, this is why lots of specs in the past have tried to separate very clearly the business semantics from the technical semantics, such that there is absolutely no confusion. Unfortunately REST encourages, at least doesn't discourage, people to define there own business meaning to an HTTP verb which really means SEND Message with a series of attributes that the infrastructure understands.

@Adam, because Roy's thesis is less than 30 pages? when you combine it with the HTTP spec, plus a couple of books to understand really was meant by the thesis and the HTTP spec, I am not sure the comparison is that advantageous.

Anyways, I am not defending any technology, I am a metamodeler, I help people abstract from the technologies and architectures they use, because, they change and disappear all the time. I thought I had written that blog n a technology neutral giving 3 simple rules that anyone can follow, be it hard core RESTafarian or casual API provider. But I guess not.

>> How do you express an XMPP interaction in REST?

Why would you? Ignoring the category error (REST is a style, XMPP is a protocol), XMPP is based on a different style. There are attempts to move XMPP over HTTP (such as BOSH) that are unwieldy, but that's to be expected due to the clash of styles. Otoh, HTTP has greater reach than XMPP so I expect this to continue, while ignoring /why/ HTTP has greater reach.

>> Yet XMPP is extremely common in the mobile world,

Probably not as common as HTTP and unless XMPP is optimized to some extent, probably not going to see continued growth. I like XMPP, but it predates the same mobile systems you say REST/HTTP isn't suited for. XMPP (more specifically its style) also has significant scalability challenges where receivers/subscribers outnumber sources (I'm talking about C2S not S2S - XMPP federates quite well). This probably isn't visible in the enterprise domain where there are just a few hundred/thousand employees at most but even small scale consumer apps can suffer without a very solid systems design, and some have backed away (Google with PSHB/C2DM, Twitter with BirdDog, possibly Nokia - bit.ly/mUQWYp). Otoh scaling HTTP traffic is well understood, and REST architecture helps there.

> As I said, there is no superior uniform interface and
> all uniform interfaces can pretty much be reified on
> top of any uniform interface.

Right, no-one's arguing at that level of architecture that one method set works for all domains. Specific standards (SQL, HTTP) do mandate a core set, and HTTP is clearly extensible. But domain specificity leaves intermediaries in a pickle as they need to understand all current and to be defined application message semantics at layers below the application - that is not a forwards compatible systems architecture. The industry that grow up around SOAP ended up creating a sub-industry of middleboxes to solve this problem (including security boxes). HTTP's limited method set for better of worse has allowed it to scale to planetary levels in part because proxies became a commodity - for all the variance at the application layer you identify, they all pretty much can use commodity intermediaries. This is a huge systems advantage that often seems to be lost on the enterprise domain, which is still churning out expensive bus architectures to mediate interchange. But even protocols like WebDAV have gotten this wrong by failing to define the cache behaviors around methods PROPFIND. A protocol that allowed you to derive more specific (comprehensible) verbs for the app layer but whose primitives could be classified by the transfer layer is an interesting idea, efforts like FIPA vaguely overlapping with the OMG tried this, but didn't gain much traction.

> In the end nothing is a problem, the only problem is the time it takes to build something,
> and we have gone from an era where it would take about 2 minutes to consume a Web service
> to now where it takes anywhere between 2 days and 2 weeks.

I would think you'd care about maintainability, but core WebServices technology like XML Schema seems spectacularly bad at evolution compared to JSON or code on demand JavaScript. The way I've seen interfaces take 2 minutes to derive Schema/WSDL is generate contracts from code, but I have a sense that's not what you're advocating.

> Can you tell me how much we won or lost going
> from machine readable to human readable?

You've set up a false dichotomy. XML Schema/WSDL/SOAP isn't machine readable in any interesting sense compared to a JSON/WADL/HTTP alternative; none of the de facto technologies provide this beyond the trivial tooling concerns. If machine readable is valuable to you, I'd be surprised if you're not a fan of technology like RDF that supports logical inference and a formal semantics. In my mind machine readable applies to the data, application and systems concerns.

> The problem is when you create hundreds of Web APIs, for a specific domain, say,
> mobile applications, the question becomes is the uniform interface you chose adequate?

I agree there's a problem but am less sure the problem is with uniformity than with the concept of an "API" itself rather than a contract. There are particular limitations I can identify with REST/HTTP, but uniformity has not been not one them.

Bill,

thank you for your thoughtful comments. I don't think there is much to agree or disagree. It is the usual, I say, you say. The only difference from 3 years ago is now "we see". There are 3000 public Web APIs published, less than 1% is RESTful. 99% of them break pretty much every single REST principles. All of them have carefully crafted proprietary ways to use HTTP which makes it a "kitchen sink" protocol thanks in part to 3-legged scenarios. Something is missing in HTTP? no problem, just add a custom header.

>> I'd be surprised if you're not a fan of technology like RDF
Sorry, I am just a simple guy, the only thing a bit fancy I do is metamodel oriented programming (and I was teaching this kind of thing back in 93, so most people would consider that outdated). Looking at the world's leading programming models in mobile and beyond, it is easy to see how to gain 90% efficiency in software development with little effort. I don't need any semantic goop, at least just yet. I need to remove all the boiler plate code brilliant architect throw at programmers.

HTTP is the Achilles' Heel of the Web. Much more efficient, secure, modern, protocols are coming, make no mistake about it. An HTTP request, is just a string...

For my part, I am glad that we are getting beyond the mostly fruitless soap vs. REST discussions to realise that we do need service interface descriptions (nee contracts).

If REST has not happened in over 5 years, it is not likely to ever happen. We should get over it and move on to discussions of how to be build better service interfaces. That will be much more useful.

There are many questions still on the table as to:
a) How to best describe service interfaces
++ human readable only
++ structured metadata (WSDL, oData, Protocol Buffers, OWL/RDFS, ...)
b) What protocol(s) (HTTP, Soap <over some base protocol>, Sockets, AMQP, ...)
c) What serialzation formats (XML <possibly binary>, JSON, RDF, ...)
d) Secrity concerns (authentication, signature, encryption, assertions, ...)
e) Transactionality
f) Asynchronous communications (with associated state management and callbacks)
e) Events (pub-sub)
g) How much toolng do we ideally need

WS* at least has an answer for most of the above (although it may not be the best answer).

We do need answers, though. And, collectively, that will be a much better discussion.


</possibly></over>

Jean-Jacques,

There are 3000 public Web APIs published, less than 1% is RESTful. 99% of them break pretty much every single REST principles.

All of them have carefully crafted proprietary ways to use HTTP which makes it a "kitchen sink" protocol thanks in part to 3-legged scenarios. Something is missing in HTTP? no problem, just add a custom header.

HTTP is the Achilles' Heel of the Web.

Much more efficient, secure, modern, protocols are coming, make no mistake about it.

An HTTP request, is just a string...

thank you Faisal. Exactly my point. With billions of (native) mobile apps downloaded each year, hungry for power consumption, bandwidth, security, privacy and monetization (as in in-app payments for instance) with the enduser attention going away entirely from Web, we need protocols that adapt to this new world of Converged applications.

This is not your old IDL. We need a massive overhaul, as you pointed out.

HTTP is antiquated, I know that a lot of people are emotionally attached to it, but its over. HTTP is the new CORBA.

>> To do anything useful, programmers have to use technologies like SVG or the
>> HTML Canvas tag and basically build from the ground-up graphics
>> abstractions for interacting with users.

John, what is key to understand is that the problem is less a technology or architecture problem. The problem is that when people "code" something they build a solution. The solution itself can be described by a model that is independent of the technologies and architecture used to build the solution.

It's great that vendors build these technologies. Fundamentally there is nothing wrong with any of them, they do their job to a certain extend. Some can be better than other. It does not matter.

The real problem of software engineering is that you cannot build a solution without tying your solution to the semantics of the technologies and architectures that you are using, be it REST, SOAP, JavaScript/HTML or JEE, SQL, NoSQL, ... The higher the mismatch (for instance between exchanging business messages and HTTP verbs), the more divergence you will see amongst developers. Programmers are not "bad", they have a solution to build, finite amount of knowledge and time, they have to make decisions about reifying the solution into the technologies and architectures they have chosen. That is the problem.

Software engineers have to understand that we need to let vendors build technologies and architectures, but we need to build solution models that are independent from these technologies and architectures. This is the decoupling that needs to happen. We don't need semantic technologies, at all, they will never work because they look at the problem the wrong way, they are trying to be a band-aid on an already completely flawed approach.

It shoudn't even be a question when I build some software whether I use REST or SOAP or whatever. The answer is, "it depends" and we propably will choose on criteria that have little to do with the semantics of the solution we are building. They will be mostly non functioning requirements.

REST has failed in a big way, it has once more set the mindset of generations of developers to a late seventies, early eighty mindset. The reason it has failed is becaused people picked a technology HTTP and claimed that it was at the level of a solution model. SOAP had at least the merit of trying to actually express the solution model, but hit the wall each time you had to stich a technology to it. When we realized that, that didn't work, we killed the solution model that SOAP had brought, and replaced it with a technology, instead of continuing to expand the solution model to the point where technologies and architectures are just an implementation detail of the solution.

Of course, few vendors will ever open the door to abstract themselves out of the picture, yet, there is no other way, no technology will ever be built that much the semantics of all solution domains (business, games, content/media, data, ...). We'll be circling like this for another 100 years if we think we will solve this by yet another "technology" and/or "architecture"

Our industry is ripe for a major overhaul. Make no mistake, it will happen.

Jean-Jacques,

I think you are conflating arguments here still.

I'd recommend writing your thoughts down, have a few people you trust review it before you publish, and then submit something concrete. Try to avoid conflating arguments. Do not conflate business concerns with technology concerns.

As for what you call "Converged Applications", I'd recommend you read Mark S. Miller's highly readable Ph.D. thesis titled "Robust Composition". Mark is featured on InfoQ often, usually as it relates to securing JavaScript and improving browsers, and he is one of the pioneers of the Internet. Based off his work and others, Tyler Close created the Web-Calculus. Most of the ideas in both come from "agoric computing", which is an approach to computing that tries to use market mechanisms to reason about scarce computing resources.

The major benefit of reading Mark's thesis is that you will get a much better understanding for what it really means to provide security while addressing monetization. Also, it addreses needs you mentioned earlier, refering to building composite applications, because the ideas address secure interaction involving more than two principals, a necessary requirement for any real world composite application. Real world composite applications are dynamically distributed, dynamically federated, and security concerns need to address that directly.

Cheers!

Of course, few vendors will ever open the door to abstract themselves out of the picture, yet, there is no other way, no technology will ever be built that much the semantics of all solution domains (business, games, content/media, data, ...). We'll be circling like this for another 100 years if we think we will solve this by yet another "technology" and/or "architecture"

Our industry is ripe for a major overhaul. Make no mistake, it will happen.

John:

you are free to believe so, but I have published a number of papers [1,2...] detailing what I am talking about, even started building a tool in that direction. Clearly this is not the place to have a discussion about this topic. I do, however, stand by what I said.

The thesis sounds interesting. I'll certainly take a look at it.

JJ-

I don't think I am the right audience for the papers you published. Sorry.

As for Mark's thesis, it should appeal to you. I just noticed your background is in financial institutions. You should be able to readily see how applying market mechanisms "just works". But if you have questions, feel free to e-mail me or ask on the cap-talk mailing list. Everyone is a newbie at some time or another.

John,

looking at your profile on Linked In the the description of your company "Breuer & Co. provides software and service solutions for education and health care. We serve organizations who want highly-tailored solutions with an extraordinary level of support, but with the same cost and delivery times as traditional packaged software.". You must have an opinion or two on how to build "highly-tailored solutions" in a technologically complex environment. Did you guys publish anything about how you do it?

Jean-Jacques,

Did you guys publish anything about how you do it?

Am I correct in understanding that this article advocates that REST become a lot more SOAP-like? I think the critical problem not addressed by the post is why REST is so popular.

1. REST has a low barrier of entry for supporting languages.
2. REST allows the service provider to determine the response type (not married to XML).
3. REST is a better paradigm for resource-based access.

There's no reason why a REST provider couldn't use XML + Schema to define and version the model. Alternatively, there no reason why developers couldn't create their own formats with their own schema mechanism.

I'm sorry that REST consumption requires more effort than pointing Visual Studio or Eclipse at an endpoint and auto generating the scaffolding to call the service. What blog failed to mention is that most vendors provide native clients to their own REST or RPC API's (and not just in enterprise languages, but also JavaScript and PHP). In those cases, the act of using REST is actually simpler than using a nasty SOAP consumption tool. More importantly, there's an assumption that code generation in SOAP is perfect and that you never encounter problems when using SOAP.

There are very real reasons why people don't use SOAP. This is not to say that SOAP is bad. It has different use cases. In my opinion, the one thing Jean got absolutely right is that you are going to need to create a consistent domain model across your services. Otherwise, what's the point? You're using creating a Service-Oriented Architecture (emphasis on being holistic). If you treat your enterprise as a collection of stove-pipes that speak different languages, no communication technology is going to save you.

"Don't be shy, forget the 4 little HTTP verbs, you can even define your own: Query/Do/Notify/Synchronize is a perfectly good one. It means that all methods in your Web API will be of type query, action, notification or sync request."
REST does not define what verbs should be used. It's HTTP that does that. Defining additional verbs and media types for a particular problem domain is a perfectly valid application of the uniform interface constraint. In the upcoming "SOA with REST" title we clearly identify governance of the uniform contract for a given service inventory as a key responsibility within an overall inventory governance framework for REST-style SOA.
What REST does do is require that any interactions supported by the verbs of the uniform interface also be stateless. That's a constraint that can rapidly become less important in smaller architectures than that of the Web.

"Explicit message definitions which can be explicitely versioned is essential to a healthy API definition. Message Extensibility is the property that makes distributed computing work."
Everything gets explicitly versioned, yes, but old implementations should be able to work with new implementations. There is a significant amount of benefit to be had from ensuring both backwards-compatibility and forwards-compatibility. It means less upgrades are required in general, that upgrades are less urgent, and that you get into situations where you have to upgrade the entire world's systems in one night much less frequently ;)

"A uniform interface is not enough to form a contract, it is only the substrate on which a contract is defined." JJ believes that the only way in which to build a "healthy ecosystem of Web API consumers" is to use machine reable contracts that are versioned to ensure evolution and reuse.
As we discuss in SOA with REST, service contracts are no less relevant in REST than in conventional SOA. However, service contracts are defined in terms of uniform contract methods and media types. The uniform contract focuses on how information is exchanged between services and consumers (the methods) and what kinds of information are exchanged (the media types). It is abstracted away from any specific business context in which you might want to transfer a particular type of information in a particular way. The service completes the semantics of any particular interaction by defining specific URLs for which a specific method will have a specific effect, and for which specific media types are understood.
In REST we try and minimise the amount of service contract information that makes its way into the consumer's logic. We do this with the objective of reusing the consumer as well as middleware components in conjunction with multiple different services that all support the same uniform interface methods and media types. If a particular consumer only ever wants to talk to one service, that's fine. It can embed whatever contract information it wants. However, we expect to be able to support generic consumers and middleware that is not locked in to only interacting with a single service.

Nice response, Bill. Pretty much captured my thoughts, exactly.

It is amazing to me how many (most?) still don't GET REST or SOAP or ... I could go on. Everything MUST be a panacea or it is crap. I have a hammer, everything MUST be a nail.

REST != HTTP and HTTP != REST. HTTP was informed by REST architectural style, but it also has some features that are inherently un-RESTful. JJ's "unREST" is not un-RESTful (necessarily) it is more a rant on HTTP. Roy's thesis does not prescribe GET, PUT, POST, DELETE. These verbs aren't even mentioned in the definition of REST for exactly the reason you cite. Because it is not a protocol, REST is an architectural style. What JJ does get right is that Roy specifically said that REST is not a panacea, that it is optimized to a certain class of system. However, that does not mean that if you are not architecting the class of system for which REST was conceived, that the REST architectural principles cannot be applied.

Anyone who hasn't read Roy's thesis multiple times needs to do so before commenting on this thread. The same applies to SOAP. It is important to understand WHY the principles of REST enable a global scale internet, and then to apply those principles/constraints in the design of a distributed system, pragmatically.

There is nothing inherently wrong with SOAP (I make no excuses for WSDL;-) - if only the tools and runtimes had not internalized the RPC-centricity of the original SOAP spec from Microsoft, we might be having a very different discussion. For that matter, the principles of REST can indeed be applied in machine-to-machine distributed computing scenarios - but that does not mean one must use HTTP.

- Chris

JJ, REST has not failed. The industry has failed REST by hyping it all out of proportion, and distorting its core messages.

The problem is sheep, and no protocol, no architectural style, no magical tool will solve that one.

- Chris

Hi Chis,

While I agree with what you state, I have to point out that in practice REST ends being very difficult to achieve.

I believe there are 3 things at the very core of REST and if you deviate from even *one* the whole thing falls apart:

1) Uniform interface to resources (protocls, verbs, etc. do not matter)

2) Late binding and standard media types (clients, when they receive a resource representation, should 'discover' what it is and it should be one of the known standard media types that any client can handle in a generic way)

3) Hypermedia (clients can easily construct links from the content and drive further interaction)

All of the above give you a 'contract-free' eco-system.

The problem lies with #2 which is very hard to achieve in practice. As a result you don't get a contract-free eco-system.

All of interactions today (that I know of) are (implicit or explicit) contract based.

Regards,

Faisal

Chris,

thank you for chiming in, just to be clear, and I agree with you, what has failed is the set of arguments that was sold to the industry shamelessly since 2007. REST itself, as defined by Roy is obviously not a failure.

My strongest regret is that a lot of progress, of which you are a major contributor, had been made in distributed computing / composite systems since the inception of XML (obviously not as a syntax, but as an extensible message structure). All this progress has been compromised for #no reason at all#.

It is time that we get back in the right direction. There is probably too much at stake for that ever to happen, but one can hope.

Chris,

don't you think that in essence nothing will change, including applying REST principles pragmatically, until the underlying programming model evolves? Isn't the core problem of build composite systems that we try to create them as a sum of monolithic systems? Do you really think that the problem is really a better way to exchange messages?

SCA had opened a very interesting way by helping change programming models in a non intrusive (and very smart) way. I am sure Jim Waldo must be laughing at all these discussions. He is confident we'll still have the same discussion in 20 years.

I kind of agree with some things and disagree with others. Concepts are always confusing in a discussion, if they are understood differently by each participant.
History:
1. Web Services was a good idea. The worst mistake was to use SOAP, as it was a clear distributed object artifact. Adjustments to SOAP to allow document style (that were actually ignored and tools created workarounds to use document by implement RPC) did not help.
2. Web Services started to be used for things that had nothing to do with the appropriate application of the concept. Using web services to communicate components in a distributed application is plain dumb. Yes, I know many will ask why. I ask Why people may want to use a web service when they won't need decoupling but need speed and reliability. Well, they do because it was a hype and we need to be in there.
3. REST makes perfect sense for web applications of the "networked type", with data centric requirements and oriented to transfers of hypermedia. Any other use, like proprietary data or processing intensive applications make no sense for REST. But many people will follow just to "be in".
4. 95% of REST discussions are about HTTP and how to use it to get REST in the correct way. Almost no architectural discussion.

So, I can say that the Uniform Interface concept may not be totally understood, and that REST goes beyond 4 operations. The same I can say we may need web services not using SOAP, and that REST web services are NOT that option. Actually, REST Web Services had some impedance mismatch problems, and I commented to InfoQ that REST should Not be discussed in the SOA channel but on Architecture or have a channel for itself. REST has nothing to do with services or SOA.

Oh Well, nice to see discussions, but I would argue we need to see actions. I'm working on DOSE, a truly Document Oriented Service Engine, which actually goes beyond the server and thinks on the client architecture as well, including the integration with the business domain (because services are business functionality) while avoiding the Forced Domain and Forced Paradigm antipatterns.

Also, Mike Amundsen is working a lot on the client part of REST and the hypermedia semantics, which is a very valuable work. REST Client architecture is something I'm also interesting in working a little bit.

In any case, until we have something else, I guess we may need to go with REST or SOA (using SOAP) to create our urgent apps.

Regards.

William Martinez Pomares.

I recommend everybody on this thread read the article by Sarah Sheard, Life Cycle of a Silver Bullet.

www.crosstalkonline.org/storage/issue-archives/...

I feel that REST has been hijacked by marketing people of consultancy selling the latest silver bullet to all your enterprise api problems.

Oh, no, I'm perfectly aware of the Fielding thesis; it could be proven that SOAP, when used well, is "RESTful" in the thesis' original sense.

That is not what we call REST. It's something everyone thinks to know and noone implements anyway.

What I claim is, that the underlying problem with REST is that programmers are lazy to understand: they are lazy to understand SOAP, and Google throws some HOWTO-s on REST, which is not too much more than a quick-and-dirty solution.

SOAP is the theoretical successor of the Sun-RPC, COM/COM+, CORBA line of services. REST would be the theoretical successor of the HTTP-for-HTML.

However, people need an RPC protocol for information retrieval mostly. For that, REST can make sense. GET, URL? Fine! Once you start to have really large datasets and multiple filters however, things get confusing.

And once you start to actually write these large datasets, things get crazy.

We failed to provide a general solution. SOAP would be one, but it's too far from the average dirty hacks. Actually, it's a misinterpreted protocol (being document centric with its envelopes and XMLs, instead of procedure-centric).

There are other experiments, like the AtomPub (Google GData), but anyone who has ever used it with a spreadsheet knows it's just wrong simply.

If a new protocol would arise, it would be JSON based for sure. I really miss a WSDL for JSON which would be able to provide us a coherent description.

Google tried it this year, but AtomPub is not a format to easily do RPC.

Also, we would need lazy loading, and perhaps full marshalling (so, functions also!) Example would be, to get an Artist's main data by its ID, and the scheme definition would allow to say getAlbums(), or getTracks() with providing filters. But perhaps it's not a responsibility for a stub.

We may need protocol-mixins, OAuth is the finest example, but I'm pretty sure, in enterprise systems, we would like to define a standard protocol for two-phase transactions and other WS-I* stuff, which are lost / custom implemented today.

We must never forget, that such a protocol is written for untalented, unmotivated, and nowadays uneducated people, enterprise programmers, and self-proclaimed web developers. We must be able to provide something which is roughly understandable from a single tweet, and documentation shouldn't be much longer than a single A4 page to make people feel they can provide services / clients for such. It's a hard job, but if we fail, simply status quo remains - as we failed to kill ad-hoc dirt with both SOAP and REST so far.

Thank you for all these discussions, I'd like to sum up what I got out of it. I noted that we are certainly at a turning point in REST:
a) There are far fewer people that are afraid to say that REST doesn't work. 4 years ago that was pretty much just me
b) all the hardcore RESTafarians are left to point out "look HTTP is everywhere" so why do you even /#!/complain
c) The only reasons they can find to explain why 99% of public Web APIs are not designed following RESTful principles is because developers are "bad", "lazy", "uneducated" ...

The fact and the mater is when 99% of Web APIs are not RESTful? The fact that all these APIs use HTTP in inconsistent way from a security or semantics perspective is of course of no concern. How would you honestly call that? a failure? (Chris, I am not sure we can call that just a failure of the hype, the number is at least 99%)

I’ll stand by my conclusions
a) pick a better uniform interface (specially in the light of mobile 3-legged scenarios which will be the overwhelming # of converged apps that will be produced in the coming years) -> HTTP's uniform interface is abysmal
b) define machine readable contracts that can be versioned explicitly -> they are actually much more human readable anyways
c) there is absolutely no reason to "couple" the exchange of a message to "resources" via the URL path, that does not work at all, a message is a message, period. -> Define your message types explicitly as part of your contract.

Hi Faisal,

Agree, #2 is difficult, especially when one takes the media type square peg and tries to hammer it into a round hole language or object binding. :-)

The other main challenge is dealing with version skew, especially in the context of the square peg/round hole binding problem above. Hence the need for some form of "contract". However, I think the real problem with #2 (especially in the context of application/xml) is that developers don't like XML, they find it clunky because they don't like DOM or have difficulty dealing with extensibility and versioning. Of course - this is also because the systems they write, and the programming languages and tools they prefer do not deal well with dynamic versioning and extensibility, either.

Maybe JJ's point is not that REST has failed, so much as REST has not lived up to the hype that it solves world hunger and world peace in one swell foop, and that it enables mere mortals to develop applications that scale to the level of Twitter, FB or Gmail. The fact remains that it is STIll hard, and will remain a significant challenge to develop such systems. REST helps enable such systems, it is not the complete answer in and of itself. However, if one's understanding of REST is limited to the belief that REST == HTTP, then they are more likely setting themselves up for a bumpy ride.

- Chris

I'm sorry but I may disagree with you here:

c) there is absolutely no reason to "couple" the exchange of a message to "resources" via the URL path, that does not work at all, a message is a message, period. -> Define your message types explicitly as part of your contract.

Adam,

>> A resource should be accessed by standard HTTP
>> Resources are identified by their URL

REST is about exchanging resource representations, quoting Roy: "A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace."

InfoQ needs to back date this post to 2009 and leave it there.

REST is not the end-all, be-all. The fact that this article postulates that it is so, nullifies every premise it attempts to make.