An API doesn’t have a natural user interface which makes it a bit harder to work with, Ben Barnard and Felix Leipold, both developers at Nokia in Berlin, explained in a presentation at the GOTO Berlin Conference when sharing their experience designing and building a public web API in their talk "Building, running and promoting a public API".
The API they are working on provides an ability to search for places e.g. hotels, find places around locations and similar geo-location functions. It’s a frontend API retrieving and aggregating data from several backend systems, and presenting a unified view to the clients. It is used both by internal and external clients.
The first thing Felix pointed out is that an API doesn’t have an obvious user interface. If you want to know what is actually going on, e.g. when there is a problem, then a request is quite an abstract thing. You have the HTTP headers and potentially a body and when the response comes back you again have the HTTP headers and body. All their responses by default returns JSON so to be able to view the responses in a user-friendly way they added content negotiation, when a browser makes a request they will instead return html. They later made a number of forms, basically one form per endpoint, to be able to enter data for a request. Ultimately they ended up with a full UI which they call Playground, a tool to simplify for developers to check the API or just play around to understand how it works.
A technique that didn’t quite work out as intended, was in testing to ensure they had backwards compatibility; they have to support clients that cannot handle an API change and therefore need to ensure they don’t break clients that already are calling the API before they release a new version. One of the ways they tried to do this was by using Consumer-Driven Contracts, CDC; the idea is that clients write a small application according to their expectations of the API which they could then use in testing to verify the API. But their experience shows that it’s too hard to get clients, both internal and external, to deliver these applications they need. They also had problems with false positives, with failing tests when there were no breaking changes, as well as false negatives, where changes that should break the client didn’t. So in the end CDCs didn’t provide the confidence they were hoping for.
One technique that has provided more confidence is using information from the production access logs to test whether a new version of the API is going to handle the kinds of requests that they see in production. What they do is taking URLs and request headers from the logs and uses a tool to replay the requests against both production and a new version of the API, then comparing the responses from the two requests. They do the comparison in two different ways, one manual check of any differences in the returned JSON, one automatic check of the response codes.
A challenge Felix described is how to follow an incoming request through a number of internal requests to backend servers to returning a response. To solve this they have introduced an extra parameter attached to the requests which make everyone handling the request either by doing extra logging, alternatively by adding extra information to the end of the body which the front-end server can then retrieve and log. This technique has proven to be incredibly useful.
The GOTO Berlin Conference 2013 is the first GOTO conference in Berlin, with about 420 attendees and 80 speakers.