RESTistential Crisis over Hypermedia APIs
In discussing his "RESTistential crisis", Cordell noted that the REST style has finally started to reveal its best hidden secret, the hypermedia constraint, after years of debates and practice. While perfectly adapted to human-driven interaction as demonstrated by the Web, there appears to be growing concerns within the web API community about its usefulness for programmable web APIs in general.
He started the conversation with a description of REST and hypermedia limitations when applied to domain-specific web APIs, then considers the need for alternate architecture styles, taking the best of REST but replacing some constraints to obtain different benefits, such as efficient and reliable machine-to-machine communication.
The main concerns cited mention the various ways to deal with changes of an interface over a long period of time. The first solution requires running several API versions in parallel and even several orchestration APIs dedicated to specific client experiences as illustrated by Netflix. Its flexibility comes with significant design and operation challenges.
The second solution requires putting much more effort upfront by designing a RESTful hypermedia API that would more easily cope with changes, limiting the impact on the clients, in a way similar to how HTML browsers and servers work and evolve. But some participants, especially web engineer Mike Kelly, voice concerns about the complexity involved and additional burden on API consumers.
Here is a summary of the conversation including large excerpts of Evan's initial message including section titles:
1) A good API doesn't change
- Thinking about Joshua Bloch's experience regarding the design of public Java API, it seems valid to say that the same rules apply for programmable web APIs.
- APIs are interfaces, following the facade pattern, that shouldn't change on the surface for clients. However, multiple or evolving implementations including the runtime data can be exposed, without requiring the API contract itself to change.
2) Versioning doesn't work
A "Web API" exposes far more than a "traditional" API. It exposes data, and more often than not it exposes the current state of a large set of constantly-changing data. How can you effectively version when your domain model has two versions, which share some subset of data, but are accessed in different ways? You force the clients to break.
- Hypermedia can help an API being less fragile and evolve, but if the data and domain model fundamentally changes. Clients won't be able to automatically adapt to all those change without human intervention from a developer.
3) Hypermedia destroys resource location
- REST prevents fixed URIs and any prior knowledge from clients about the server resources. In practice clients remember URIs (e.g. bookmarks) and servers need to deal with evolution of their URI space (redirections).
- Hypermedia can't isolate the API providers from the laziness of API users who will not follow the proper hypermedia flow but instead will directly target specific resources and URIs.
- As hypermedia APIs tend to be very complex to understand and consume, developers will inevitably optimize their clients and take short-cuts, defeating the whole point of isolating clients from API changes.
4) Hypermedia make sense for human users, not machines
- Machines don't deal well with change over time but humans are very good at interpreting changes and deciding what they want to do next. Hypermedia is very natural to humans, much less so to machine.
- Changes in the domain model require changes in the way it is consumed by clients, hypermedia doesn't buy you anything special here.
A true REST/hypermedia client is really a form of HCI, rather than an API (an AHI - Application Human Interface, if you will)
5) What's so bad about out-of-band?
- Does an API really need to be inherently discoverable?
- Why should the API be self documenting, when the format of the API responses are difficult to read at high levels of complexity?
- Nicely structured HTML, rendered by a browser, with interactive examples and articulate descriptions is easier to read.
6) Are hypermedia APIs really that different from a WADL?
What's so different about a discoverable, hypermedia API? A "HAPI" just seems like an API with the WADL distributed throughout. You still have to start with a single endpoint, and you still have to figure out what you can with the API based on that response.
- Something has to have a description of the domain models, and that's something that will change over time.
7) A Web analogue
- REST in native mobile apps would require Android ML and iOSML languages with code on demand.
- Use of domain specific hypermedia media types still requires you to write custom client code that understands this domain.
8) What seems to really be good about REST
- Beside hypermedia and the Web of documents, REST:
- enforces a stateless server, making it very easy to scale
- encourages decoupling of information via resources
The basics are so simple that you can talk to servers in a semantically meaningful way with nothing but the http library included on every platform and language.
The above summary contains several excerpts from the participants, especially Evan Cordell, and also Mike Amundsen, Jorn Wildt and Mike Kelly among others. As a disclaimer, note that I also participated in the beginning of the conversation.
Based on your own experience do you think we need to better differentiate the REST and the web API architecture styles, as well as between human-driven hypermedia and machine-to-machine communication use cases?
I am totally flabbergasted to see that it took some 7+ years the REST community to discover/admit that and some "limitations" of the REST style. They look pretty self-evident to me...
Give it another few years and the REST community might serendipitously "discover" how to do versioning properly.
In the end there are 3 established styles of communications between systems:
- Object Orientation
- Service Orientation
There are pros and cons in using one or the other. After all, we've been CRUDing databases for as long as there were RDMSs. The big flaw with an Object Oriented/Resource Oriented/Domain Oriented approach is that "objects" are accessed in context. What I mean by that is you don't access an employee data the same way, whether it is part of the "contact context", the "benefits context" or the "payroll context". Sure, in trivial cases there is a one-to-one relationships between context and resource, but one shouldn't be fooled by it. In most cases, you need to put the context first and the resource second, that's why service orientation was invented. But, what do I know? Mike Kelly? Mike Amundsen?
So if I have an employee resource with the URI /bob, then I can have a 'contact' link to /bob from an address book resource, or a 'payee' link to /bob from my payroll resource, or a 'beneficiary' link to /bob from a benefits resource.
I am not quite sure I follow you, the "rel" value is not used in any way when accessing the resource, this is merely an agreed upon (aka shared understanding) semantics that is used by the client to identify specify relationship it is capable of acting on.
If I have:
<link rel="contact" href="/bob"/>
I can't also have and expect a different resource representation for:
<link rel="payee" href="/bob"/>
The context is set in the way you access the resource, not via the rel value.
I know that most of the RESTafarians explained in the past how the rel attribute was automagically interpreted by the client, but that's pure fantasy. A user can do it, a machine just can't.
In the end you have to decide how you access information: in a service oriented way or in an object/resource oriented way. CRUD is pretty much a non starter in most cases.
Service Orientation is IMHO the right way to access information as it provides a true unit of definition, management, versioning, ownership, authorization...
More Objections to HATEOAS
When two systems communicate you need a great deal of "shared understanding" and establishing that shared understanding is not cheap. That will probably remain true until AI materializes.
A truly restful architecture already has a great deal of shared understanding. CRUD operations, json, and relationships in the context of HATEOAS being examples.
If you mean that it requires a great deal of information shared out of band, then you're pretty much wrong for a great deal of the use cases of that API. (a great deal being the vast majority).
The idea that a machine just can't interpert a hateoas relationship makes no sense since I can just program it to recognize the rel by making templates that react to it, and ultimately stamp out forms/html that do what I want my smart client to do.
Your example is a bit misleading. Why would you have a 'bob' resource? Bob is an instance of a person, not a Bob, and out of band data anyway. Your client doesn't care. Talking about out of band data only serves to confuse how a HATEOAS client actually works.
There's nothing automagic about it at all. You program your client and work out the bugs. It responds the way you intend it to like any working program. You see a rel link going to a resource you know the associated ID stored in the hateoas response can be used via REST against that resource to perform any CRUD operation. You have the option in your client's templates to use the ID/Resource to build forms that perform the CRUD or you can ignore it if you'd like and only build forms using the direct attributes of your json object.
Things like Bob and Person only apply at run time because your actual code is running purely on the metadata of HATEOAS and your templates that are also runtime editable data, not out of band data you code against. Once up and running you can drop every table/model on your back end, design an entirely new database/model structure, and your entire client will adapt and "just work" instantly. No regressions. No client pull requests. No client code reviews. Not to say there aren't limitations where better tools can be used for edge cases but you can easily eliminate 80% of rote work. (guesstimate)
It's incredibly powerful, but not magic.