Article: Subbu Allamaraju on Describing RESTful Applications

Subbu,


Nicely written article on REST. Your bank.org examples clearly explain how to improve a RESTful design to exhibit full self-describing contracts.


I agree with you that the lack of a universal contract language could be a show stopper for some developers given that it increases the amount of manual development required to bind to the RESTful interface. Variances in RESTful contracts between vendor APIs also diminishes the number of universal tools that can be created to ease the binding for client software. A lack of a universal contract language for REST also means that there is a burden to write code to construct the contract and distribute it through HTTP OPTIONS on the server side. However, given these limitations, the approach still has its positives based on flexibility and simplicity.

There are a couple of REST-like frameworks. In Java there the free, open source framework called Restlet [1] that now nicely integrates with GWT. In the .NET world, Microsoft has created an free framework called ASP.NET MVC [2] that embraces the power of HTTP and the URL. However, for Subbu's demonstration with his RESTful self-describing contracts there is currently no framework for it (at least that I'm aware of anyway). You'll have to role your own. Without a universal way of describing RESTful contracts it hampers the ability to write consistent frameworks that work on various platforms. That being said, I believe that Subbu has laid down a great foundation that others can build upon and hopefully evolve towards a formal base that everyone can work off of.


[1] www.restlet.org/
[2] www.asp.net/mvc/

Michael:

MVC has nothing to do with REST. MVC binds URLs to actions ("{controller}/{action}/{id}"). This is clearly explained in their tutorial: quickstarts.asp.net/previews/mvc/mvc_RoutingInM...

The MVC pattern is not resource oriented by any means. At the end of the day there are several models that needs to be very clear:
- Web Services: one endpoint per service (any number of operations)
- MVC: one endpoint per action
- REST: one endpoint per resource
- Pub/Sub: one endpoint per queue

Each of these approaches allow you to associate business logic to an endpoint (they also have different activation and concurrency models). The resulting factoring of this business logic is quite different in each case.

Subbu,

This is nicely written. It felt analogous to "how can a machine-client browse"!

Is there a typo in the rewritten example under Media Types for Status? Shouldn't it be: application/vnd.bank.org.status+xml

This article is one of the few that clearly articulate what is meant with "Hypermedia as the engine of application state" - I really appreciate this.

However, I don't agree with the usage of HTTP options. Except for some "automated RESTful crawler", I don't see why any client application should do this. Except for a PUT intending to create a resource, which could be replaced by POST, you cannot interchange the verbs anyway. And even that replacement is somewhat fishy, as I assume that in most cases, the intended URL has at least *some* business value.
In that manner, changing the allowed verbs will definitively break the client code.

Wondering how you feel about link@rel described by a uri template rather than a uri. The reason is that in my system, you can create new entity sub-types runtime, then assert a parent-child relationship between them (also runtime).

It is an atompub-based content management system. A "collection" can have item types, e.g. "artist" and "artist work" (these item types are not predefined, by constructed runtime). We can then assert artist has (0:m) works -- we do so by posting a "relation" resource to the appropriate endpoint. Then we create relationships by attaching an atom:category to the child instance (e.g. on entry for "starry night" we add category <category scheme="http://example.com/typerel/artist/to/artist_work" term="tag:000123"/> (tag:000123 is the atom:id for entry describing Van Gogh).

Entry for van Gogh has <link rel="http://example.com/typerel/artist/to/artist_work" href="http://example.com?category={example.com/typerel/artist/to/artist_work}tag:00012e"/>

To document this, you would need to define a type-to-type relationship as:

example.com/typerel/{parent type}/to/{child type}

Is that too "meta"??

--peter keane

Come to think of it, as long as the documentation (or service description) can be generated runtime -- no need for uri templates in docs. Just documentation of the link@rel that points to the end point for creating new type-to-type relations. In AtomPub, I use app:categories to "document" the new relations as categories@scheme (which becomes a link@rel when used as I described above).

--peter keane

Jean-Jacques,

I believe Michael was alluding to Microsoft Astoria...

Cheers,

Zubin Wadia
CTO
www.imagework.com
"Business Acceleration through Process Automation."

Thanks. You are right about the typo. I will get it corrected.

@Michael: There is a universal contract "language" for RESTful apps, and that involves the uniform interface, media types, and link relations. A client that "understands" the uniform interface, some URIs to start with, and a given set of media types and link relations should be able to function just based on these.

I agree, and would add one more correction: The uniform interface is not just a set of methods. Document types are _part_of_ the uniform interface.

In a strictly pure REST architecture, all document types would be known by all components. Because documents on the Web have essentially one function (convey human-readable information), this is essentially possible. In environments where information has different purposes (purchase order versus employee record) the near-enough interpretation is that every component understands and knows how to generate every document type that makes sense.

The service contract listed about should not describe the media type. That is the job of the architecture-wide standardisation process for the Uniform Interface. This standardisation process should result in a set of standard methods, true, but also a set of standard document types.

What is left for the service contract to describe is the contextual meaning of any URLs that you want to "pre-publish". You want clients to know where to start from in whatever activity they are involved with. Past that point, documents should provide whatever contextual information is required to continue the client's task until it is complete.

@Benjamin:

Agree that media types are part of the uniform interface but not necessarily in the same way you imply. Uniform interface requires that media types make messages self-describing, but the interface itself does not have to describe what the media types are.

Secondly, whether media types are described via some "architecture-wide standardization process" or not depends on the nature of applications. As the developer, of say, some new set of HTTP APIs for some next big cool thing, I may choose to define the define media types myself. In cases where there are interoperating applications, some process like to describe may define those types. So, media types are part of the contract, and who defines those types can vary on a case by case basis.

@Benjamin: Just realized that you may be referring to the uniform interface in its generic sense, where as I am using it in the HTTP sense.

For Sure, this article is one of the closest to Rest concepts I have read.

I agree with it and I think the proposed solution may be usable, practical.

Still, the WSDL comments need a little discussion. In particular, two comments made me think a little about WSDL and WS perception: first, WSDL as a violation of self-describing messages and Hypermedia as an application state engine; Second, WSDL and WS as RPC-like approach.

The main idea of all this (at last to me), is that the description of the application is not out of band, but part of the application itself. WSDL has been used as a design time description to create fixed clients. But the actual idea was to have WSDL describe a set of resources called services, at runtime. Thus, if you follow, WSDL is not just a contract, but an standard media type that has all the info and links to use the application. It contains the information to create more messages and also how those messages are sent and received.

Another misuse is the RPC interaction. Services are seen as methods you need to call, when they are actually endpoints (URLs) that receive messages with an specific media type (let's think SOAP, or even the SOAP body document), and must process them. The content of the message (its type) should be enough to determine what the server needs to do with it (that is, no method name should be needed).

What do you think?

William Martinez Pomares.
Architect's Thoughts

I'm trying to find out to do in similar situations, and since I already have a "data-centric approach" like

<accounts>
    <account>
        <id>AZA12093</id>
        <customer-id>7t676323a</customerid>
        <balance currency="USD">993.95</balance>
    <account></account>
    <account>
        <id>ADK31242</id>
        <customer-id>7t676323a</customerid>
        <balance currency="USD">534.62</balance>
    <account></account>
<accounts></accounts>


I was wondering using something like


<accounts>
    <account href="http://bank.example/accounts/AZA12093">
        <id>AZA12093</id>
        <customer-id>7t676323a</customerid>
        <balance currency="USD">993.95</balance>
    <account></account>
    <account href="http://bank.example/accounts/ADK31242">
        <id>ADK31242</id>
        <customer-id>7t676323a</customerid>
        <balance currency="USD">534.62</balance>
    <account></account>
<accounts></accounts>


That way I can even deserialize the XML into a Java Object with XStream, like I'm doing now (with a different scenario than this example), and have the "hipermedia as the engine of application state" as the REST style imposes. (Ofcourse, following the link will provide me with more information than the current one, otherwise there will be no necessity of navigation, this is just a example).

What do you think of it? BTW, yours is the first article I saw that does a good job on explaining a approach using media-types in HATEOAS, it's really a great article.</customer-id></account></customer-id></account></accounts>
</customer-id></account></customer-id></account></accounts>

@William: That is an interesting interpretation of WSDL, but looking at the stated goals in WSDL spec, I don't think WSDL was designed with such an interpretation in mind.

@Antonio: That's perfectly fine, although using a link like the ones in the examples would make linking more extensible.

Stefan just pointed out that the HTTP responses with 201 response code did not have the Location response header. This was an oversight. Thanks Stefan.

Subbu

First, excellent article Subbu. Second, two typos I think.

Shouldn't this (sorry, the comment system gets rid of all leading whitespace)

<accounts xmlns="urn:org:bank:accounts">
<account>
<id>AZA12093</id>
<link href="http://bank.org/account/AZA12093" rel="self"/>
<link rel="http://bank.org/rel/transfer edit"
type="application/vnd.bank.org.transfer+xml"
href="http://bank.org/transfers"/>
<link rel="http://bank.org/rel/customer"
type="application/vnd.bank.org.customer+xml"
href="http://bank.org/customer/7t676323a"/>
<balance currency="USD">993.95</balance>
</account>
<account>
<id>ADK31242</id>
<link href="http://bank.org/account/ADK31242" rel="self"/>
<link rel="http://bank.org/rel/transfer"
type="application/vnd.bank.org.customer+xml"
href="http://bank.org/transfers"/>
<link rel="http://bank.org/rel/customer"
type="application/vnd.bank.org.customer+xml"
href="http://bank.org/customer/7t676323a"/>
<balance currency="USD">534.62</balance>
</account>
</accounts>

be this

<accounts xmlns="urn:org:bank:accounts">
<account>
<id>AZA12093</id>
<link href="http://bank.org/account/AZA12093" rel="self"/>
<link rel="http://bank.org/rel/transfer" ********************* get rid of edit
type="application/vnd.bank.org.transfer+xml"
href="http://bank.org/transfers"/>
<link rel="http://bank.org/rel/customer"
type="application/vnd.bank.org.customer+xml"
href="http://bank.org/customer/7t676323a"/>
<balance currency="USD">993.95</balance>
</account>
<account>
<id>ADK31242</id>
<link href="http://bank.org/account/ADK31242" rel="self"/>
<link rel="http://bank.org/rel/transfer"
type="application/vnd.bank.org.transfer+xml" ********************* change customer to transfer
href="http://bank.org/transfers"/>
<link rel="http://bank.org/rel/customer"
type="application/vnd.bank.org.customer+xml"
href="http://bank.org/customer/7t676323a"/>
<balance currency="USD">534.62</balance>
</account>
</accounts>

-- Nick

Now that I've addressed the typos, I wanted to make a couple of observations:
1. Your examples drove home to me for the first time that a data architect using XML and <link> now has THREE choices for representing a constituent of an XML doc: element, attribute, and now, link. For example, in the transfer document, one could have chosen to make the "note" a link to a note resource instead of an element. It would be interesting to see an analysis of which concerns would lead to the use of which style, ie element, attribute, or link.

2. I wonder why you chose to provide a transfer both an URN identifier (represented as an XML element <id>transfer:XTA8763</id>) and a URL identifier (bank.org/transfer/XTA8763). Why not just use the URL? (I think we've covered this ground before).

-- Nick


</link>

@Nick:

a. I added "edit" just to quality the link. But it does not really matter, as long as the rel name is concrete enough to tell the client the type/purpose of the link.

b. A typo.

Thanks for a keen eye.

@Nick:

1. The choice is based on granularity and access patterns. If the server decided to inline it, but wants to tell the client the client that the inlined data could be accessed directly, it can provide a link with a self rel within that inlined element. If not, it could provide just a link making it a related/child resource. Simialarly, in the case of collections, we can think of collections containing just links to members, or include a summary+link for each member, or inline the entire member.

2. You are right. We discussed this a while ago. As I tried to illustrate at www.subbu.org/blog/2008/12/resource-identity-an... the URI in the self-link does not always uniquely identify the resource. When I say identity, I am looking from the client's point of view, which may want to store representations and need to come up with a database key. This is why Atom has an <id> and RSS has a <guid>. I just borrowed atom:id for these examples.</guid></id>

re (a) either both account representations should have ""http://bank.org/rel/transfer edit" or neither should. As it is now, the 1st acct representation has "http://bank.org/rel/transfer edit", but the second has just "http://bank.org/rel/transfer". It makes the reader wonder why you are making a distinction.

re (2): One of the things I like about Google's BIGTABLE is that it was designed to use URLs as keys. Given a commitment to "cool URIs don't change", there is no reason that URLs can't be used for database keys.

@Nick: Re (2), the id is a URI, and a URL can be substituted as long as the app follows RFC 4287.

You may be also interested in looking to Project Jersey [jersey.dev.java.net/], a Java based implementation from SUN.

Fantastic article. I'm a non-IT/CS domain specialist who finds himself doing more and more programming. Your article answered a number of questions and confirmed a suspicion I hadn't yet seen a 'correct' REST description - RESTful now makes much more sense - Thank you!

Still I like some ideas in WADL:
- it is the documentation (extremely important)
- grammars
- provides an advanced starting point, even though it is not proper REST (remember my 'real-job'/domain-specialty is calling very minute)

Having said all that what you describe is very elegant and hopefully will be amenable to code-gen by some framework.

Some Questions.
In your scheme the 'rel' attribute of a 'link' element is the (only?) brittle part, i.e. if the server side code changes this data, then the client side code has to be updated (manually or automagically). The benefit of the set-up you describe is that the server code can do what it likes with the 'href' location and the client code shouldn't break, or at least not so easily. Is this a fair summary?

Next, in point 3 in your model under 'Is this practical', am I correct in thinking you'd 'create classes manually' which correspond to data in the 'rel' attribute in each link element? This 'rel' data was read in step 2 - correct?

Thanks
Mark

@Mark,

In your scheme the 'rel' attribute of a 'link' element is the (only?) brittle part, i.e. if the server side code changes this data, then the client side code has to be updated (manually or automagically). The benefit of the set-up you describe is that the server code can do what it likes with the 'href' location and the client code shouldn't break, or at least not so easily. Is this a fair summary?

Next, in point 3 in your model under 'Is this practical', am I correct in thinking you'd 'create classes manually' which correspond to data in the 'rel' attribute in each link element? This 'rel' data was read in step 2 - correct?

In your scheme the 'rel' attribute of a 'link' element is the (only?) brittle part, i.e. if the server side code changes this data, then the client side code has to be updated (manually or automagically). The benefit of the set-up you describe is that the server code can do what it likes with the 'href' location and the client code shouldn't break, or at least not so easily. Is this a fair summary?

Instead of URI in the href, the relation is part of the contract, and hence any changes to relations can break the contract. Imagine a web server deciding to use "myfeed" in place of "alternate" for a link to an Atom feed. That will all clients that are looking for that link.

Hmm, Having sketched out a client on paper I think I can now answer my own Q:
a)
Reason: Any 'insurance' contract makes no sense in REST since you cannot guarantee, from the current data, you can work out which path the client-server took to reach its current state, and that state, or at least a representation of it, may be embodied in any href's that are stale/broken.

I also think this is a very good article explaining what REST is all about.
Concerning the media types you use in the links I was wondering on how you would deal with different representations of a specific resource.
Typically you would have your account type defined by application/vnd.bank.org.account+xml.
What would you do if your service had several representations of this account, f.i. a json representation.
You could argue that once you started using xml you prabably want to continue using xml, but what if for instance you have a type within your system for which there is a standard, like iCal, xspf, ... It would probably be a good idea to also provide a resource in a standard format, like ical for a schedule or calendar. The list of schedules in your system will probably be a self defined xml type, whereas for a specific schedule you would service a specific xml type as well as an ical type.
Wouldn't it be better to just use the rel attribute and to describe for each rel attribute which media types are available in your service?
This would also make sense in the case you were building a service with several versions, i.e. your resources can have an account_1.0+xml and an account_1.1+xml representation. You could argue that once you started using version 1.0 you would continue to use the 1.0 representations. If you provide the type for each link you will probably have to have a new version for each type even if only one type changed (e.g. extra info) because all resources are linked to each other. If you would leave choosing the type up to the client, the client could decide which version he wants, choosing from the different types you specified for a certain rel attribute. If at that time he was only aware of version 1.0, he would take version 1.0, another client needing this extra info in the 1.1 version could choose this 1.1 representation. In this way you would only define one new type if only one resource had a new implementation.
Does this make sense?

I think you are overusing media types. As Roy T. Fielding stated once: "The media type is a generic processing model that every agent can learn if there aren’t too many of them (hence the need for standards)." (see roy.gbiv.com/untangled/2008/rest-apis-must-be-h...).
So why not using RDF, as a generic knowledge representation structure (resource description language), as a basis for describing media type specifications and typed links in a machine-processable way. The "type" attribute of a link (relation) is the defined by the range definition of an RDF property and the "rel" attribute is the specific RDF property itself. The description of the property (relation) can be retrieved by resolved the related URI.
Another statement of Roy T. Fielding is that "Every media type defines a default processing model" (see roy.gbiv.com/untangled/2008/rest-apis-must-be-h...); that means, a media type specification should include the definition of a process model for each link type that is part of the specification. Generally, media type specifications are only a specific kind of knowledge representation language. This, rules the application of HTTP OPTIONS out, because the set of possible methods that can be successfully applied on a specific link type should already be defined in the media type description. Since, a media type specification should also be processable by a machine agent, one can (/should) also define an RDF-based description. This enables a client also to learn a new media type on-the-fly, if wanted (analogous to the HTTP Upgrade header field).
An RDF-based description can of course include further more domain specific link types that are usually defined in separate vocabulary/ontology specifications that are reference in the description. The media type for RDF-based description should start with "application/rdf" and followed by the preferred serialization syntax e.g., "+xml" or "+turtle" or "+json".
Fully machine-processable descriptions of all kinds are a requirement for a true RESTful application, otherwise the constrains "self-describing message" and "hypertext as the engine of application state" are violated. The client should always be able to decide to which state he/she/it want to change next (cf. www.ics.uci.edu/~taylor/documents/2002-REST-TOI...). This is also quite good summarized by a further statement from Roy T. Fielding:

"Automated agents are dependent on their understanding of the media types, link types, or microformat extensions provided in representations. It is the same basic issue as with human communication: we will always need a common vocabulary to make sense of it. Exposing that vocabulary in the representations makes it easy to learn and be adopted by others. Some of it will be standardized, some of it will be domain-specific, but ultimately the agents will have to be adaptable to new vocabulary." (see roy.gbiv.com/untangled/2008/rest-apis-must-be-h...).

WSDL and WADL is for desribing services and applications. However, REST is resource oriented, so we like to describe resources in a generic way. We aren't interested in, whether these resources are implemented queues, services or applications. Cf.

"If a client is receiving instructions on how to interact with a resource just before acting only on those instructions, then the client doesn’t care if the resource is a distributed queue. It simply doesn’t need to know how the resource is implemented." (see roy.gbiv.com/untangled/2008/rest-apis-must-be-h...)

Finally, I haven't really seen yet an example of an application that completely matches the constraints of the REST architectural style. I also saw several requests in Roy T. Fieldings blog for such an application, however, he didn't responsed yet (AFAIK).

PS: "A REST API should not be dependent on any single communication protocol" (see roy.gbiv.com/untangled/2008/rest-apis-must-be-h...)

IG Group have recently open-sourced their RESTdoclet solution, which is specifically aimed at services built using Spring’s REST framework. We use this internally to automatically document all our RESTful services and publish these via an internal web portal.

More info here: ig-group.github.com/RESTdoclet/

Robert @ IG Group

Informative post - I was enlightened by the info ! Does someone know if I can grab a blank a form form to complete ?