Don't advertise your API as as being RESTful. This is one piece of advice from API developers on a recent episode of the Nodeup podcast - an APIs show. The podcast featured a conversation between Daniel Shaw engineer at Voxer and host of Nodeup, James Halliday founder of Browserling and guests Mark Cavage and Andrew Sliwinksi. Cavage is a software engineer at cloud infrastructure provider Joyent and author of the Node.js package Restify. Joyent owns Node.js and uses it extensively. Sliwinski is Co-founder and CTO of children's experience website DIY.
Kicking off with API design, the conversation meanders through key concerns of API development with Node.js, including the value of REST, security, testing, documentation, schemas and streaming. The favourite API design method is to "start with the README". Approach API design like you are building a user interface. Cavage refers to the way Amazon.com approaches product development by "writing the press release first." He adds that you want to build a minimal initial API and then evolve it based on usage experience. Don't attempt to include too many features up front.
The value of REST is regarded as mixed. The problem with advertising your API as being RESTful is that it "acts as troll bait" according to Shaw. People get too hung up and pedantic about REST conformance. More important instead is to be pedantic about HTTP because that is the common underlying protocol. Cavage's advice is that HTTP is "the interface you want to use at the firewall" but internally you can revert to whatever more efficient data formats and transports make sense, noting that "HTTP comes with a cost." He cited dnode and fast as example rpc frameworks that they use in back-end infrastructure.
The subject of API security raises some debate. Halliday and Shaw feel that new security protocols such as OAuth have become a drag on the original excitement of consuming APIs with command line tools such as curl. However Cavage counters that security must align with the value of the API you are protecting and that HTTP basic authentication is often not sufficient. Cavage introduces HTTP Signatures which he says is better than passwords but "easier than the other [protocols]". HTTP Signatures add origin authentication, message integrity and replay resistance to HTTP requests. Developed at Joyent, HTTP Signatures has recently been submitted to the IETF.
All agreed that testing should be based on real systems and data and that "mocking" is a bad practice. The problem with mocking is that your tests are detached from reality, explains Halliday. Sliwinski describes how at DIY they populate a local database with live data to perform continuous integration testing without resorting to mocks. Some of the test tools used include Nodeunit, Tap and Travis CI.
On the topic of documentation, Cavage jokes that WSDL earns "demerit points". At DIY they write the API documentation first in a JSON specification format that drives an interactive documentation webpage. Cavage describes how Joyent uses Restdown to document their APIs and keep that documentation in the repo where they host the API. Keeping documentation in-line with the API is an important challenge and all agree that they prefer to have interactive documentation with sample data.
Schemas are another problematic area. Two main problems with schema validators are that they are slow and they throw obtuse errors, explains Cavage. But on the other hand doing things by hand without formal grammers is risky. DIY uses JSON Schema but that comes with trade-offs. The standard is still evolving and there are no stand-out Node.js implementations of JSON Schema validators. Sliwinski says they've had some luck with JSV but have had to write their own wrappers to catch and reinterpret those obtuse error messages.
The podcast includes a lot of discussion around streaming APIs which take advantage of Node.js' non-blocking I/O architecture. Halliday explained that streaming JSON in APIs makes a lot of sense to avoid buffering and latency. Cavage notes that support for streaming JSON is one of the main reasons to use Restify over the alternative package Express. Caution was expressed about the use of Node.js middleware with streaming JSON. Cavage likes to use middleware such as Connect to handle "userland complexity" such as sessions and client vagaries but says you pay for that with the loss of streams. Core Node.js without middleware is best for native handling of streams.
But back to the question raised at the begining of this item. Is it safe to advertise your API as being RESTful, or does that just encourage bad behaviour from the pedants? Is REST a standard to be followed or is it a set of constraints? Where do you draw the line between RESTful and not? We'd like to hear your experience.