BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Evolving APIs for Scale with Dropbox

Evolving APIs for Scale with Dropbox

This item in japanese

Bookmarks

When building a new API, one often gets stuck between blind future proofing and a pre-optimization root evil hard place. Dropbox is no exception. When building their API, they had to account for their rapid growth as a company while also being cognizant that any changes they made to their API over time would always be perceived at least by some consumers as a net negative. Their ultimate findings? Being thoughtful is what matters.

In a blog post last year, Leah Culver, a former Developer Advocate at Dropbox, elaborated on Dropbox's struggle moving from V1 to V2 of their API. Their first big decision was whether or not to bend their existing API to fit their growing consumer needs as they expanded Dropbox Pro and core Dropbox functionality. Their decision revolved around what Culver describes as their secret to proper API growth, a "symbiotic relationship" with their consumers. The need to be able to integrate in a flexible way with multiple different company's applications outweighed the need to be entirely non-disruptive. Connectivity between multiple applications is now more important than ever. A recent Google study revealed that one in four app users discover applications via search, however according to Statista around two to three million applications are available for download in the Android and Apple app stores, making app visibility on these app store searches critical. Users are increasingly unwilling to go to multiple different applications for related functionality and missing the chance to expand the Dropbox API would mean less integration with third party applications and ultimately less Dropbox consumers.

In creating V2 of the Dropbox API, however, Dropbox went decidedly off trend. Rather than utilizing the paradigms of REST, or GraphQL or even socket servers, Dropbox created their own custom JSON in JSON out API which largely ignores the guidelines of REST or even HTTP. Rather than utilizing universal HTTP status codes, they use the 409 code for any errors with custom error messages in the response body. Their API processing layer is an HTTP POST. Rather than utilizing the request url or headers, their API takes in a JSON body and returns a JSON body, regardless of whether the API action in question is one of retrieval or state modification.

In terms of scale, Dropbox's approach has several advantages and disadvantages. On one hand, Dropbox is not constrained by the often rigid and prescriptive nature of REST, which does not fit every data use case and so is often misinterpreted beyond recognition, Steve Klabnik, RUST/RUBY contributor and author of Rust for Rubyists makes the claim that "99.99% of the RESTful APIs out there aren't fully compliant with Roy Fielding's conception of REST." This break from even the pretense of RESTful convention allows Dropbox to bend their API to whatever use case they may require in the future easily, since they are not conforming to any one set model. However, what they gain in flexibility they are also losing in structure and common developer understanding.

While HTTP status codes are a universal standard that every developer looking to integrate with the Dropbox API will understand and can easily build against, custom status codes in a response body require not only potentially messy string processing but less programatic understanding into different error states. The combining of GET and POST verbage, while providing more powerful API development possibilities, makes it difficult to determine what calls from a consumer perspective will alter object state and which are purely retrieval based, a potentially dangerous integration scenario. Many of these custom API architecture decisions lead to a prerequisite of needing to acquire a large amount of domain knowledge about the Dropbox API in particular, rather than any developer being able to treat it as just another API. In a blog post, Dropbox developer F. Metsys says of Dropbox's approach, "we're opportunistically taking advantage of HTTP without tying ourselves to it." This may mean that the Dropbox API will be able to provide features and a level of data accessibility no other API can or it could mean a confusing and time intensive integration process. Only time will tell whether the ad hoc structure of the Dropbox API is a net positive or negative in terms of their overall growth and scale.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • As always common sense over dogma

    by Bam Ezu,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    HTTP is great, and it's bad not to use its verbs when you could do so.
    But it doesn't mean you should stick to it when your API doesn't match.
    And having all your logic at the same level has value too.

  • Re: As always common sense over dogma

    by Margot Krouwer,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I would definitely agree. Each API has a different business use case and should be tailored as such. However, I do think if you are using a given protocol you should try unless there is real value not to in abiding by the standards of this protocol. One of my favorite dev quotes is "Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live." Because HTTP status codes and verbs are widely understood not only by developers but by large amounts of software by the book implementation of these standards makes the work of fellow devs much easier which provides a massive albeit hard to directly quantify amount of value to the coding eco-system. This is paramount for an API since it so directly relies on relationships between other teams and software.

    However, there are real times where these decisions do not make good technical sense depending on business needs, for example if you're working with large quantities of data in a single resource. In these cases a new solution is really needed and going to provide a long term technical benefit that outlasts a short term learning curve. I always weigh the cost to benefit ratio and there is definitely a real cost to those around you being unable to properly understand/implement your software.

  • This has been done before...

    by Michael Hibay,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Short of a few changes in the error formatting and the use of braces over brackets, why does this description sound entirely similar to a plain XML service predating SOAP?

  • Your message is awaiting moderation. Thank you for participating in the discussion.

    1

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT