Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Mike Amundsen on API Design, Governance, and Lifecycle Management

Mike Amundsen on API Design, Governance, and Lifecycle Management


1. Mike, can you tell us a little bit about yourself?

Yes. I work with a company named Layer 7 Technologies, which is based out of Vancouver. We just recently merged with CA, so we are really excited about that. My job title is Principal API Architect. It’s a title they made up for me but the coolest thing is my job description is to help people build great web APIs. So we have a thing at Layer 7 we call the API Academy: Ronnie Mitra, Alex Gaber, Matt McLarty and I work on providing guidance, gathering research and disseminating it. So, it’s all about helping people build really cool interfaces for the web.


2. Contracts for RESTful APIs have been a touchy topic. Do you think they’ll be necessary given the current proliferation of CRUD style APIs?

Yes. In a really general sense, we have a contract, even in a CRUD style API the contract is “I’ll give you a URL and then you can do these possible things with it”. In SOAP we had a much more extensive contract, we had the WSDL contract which listed a set of functions and arguments, and their types and the kind of return values. So we are used to contracts. In the hypermedia style, our contract is the type of media itself, “when I send you a message, here are the things you can do: you can create a form, you can follow a link and so on and so forth.

” No matter what we do in communications, whether it’s for machine-to-machine or humans-to-machine we definitely are going to have contracts. I think the real challenge is figuring out what kind of contract and what contracts are effective over the long term and usually that’s a contract that is not overly stringent. A contract that lets the system add new features or change features without actually breaking the contract: I think that’s the big challenge that we work on.


3. Tools like Swagger - a specification and framework implementation for describing, producing, consuming, and visualizing RESTful web services – appear to be gaining in popularity. What are your thoughts on Swagger?

Swagger - or other products like it, like I/O Docs, or which we actually integrate with at Layer 7 – these are all attempts to create documentation modeling for an API which we can present sort of as a contract. What I am finding really fascinating is as Ronnie and Alex and I work with customers, we’re finding that when we watch people work, we watch developers work, they don’t often look at documentation unless they have to. So, one of the things that I am finding very interesting is how to figure out how to create ways to communicate developers’ information without having them read lots and lots of material, so we are trying to create usable APIs.

Anybody who works with VisualStudio or Eclipse they know about system dot, they type system dot and then they get a bunch of features that’s a usable interface. When I am using your CRUD style I can do and I hit enter and I get something back then there’s links and I can follow those links and I can change the URL. So, I think all these documentation things try to solve some of those problems and I think what I find interesting are ways to reduce the amount of reading someone has to do before they are effective with your API. So, I think Swagger is filling a need, but I think there may be other ways to do that.


4. How important is affordance in system-to-system integration scenarios?

When you are talking machine-to-machine you need to create affordances that machines understand. To us a link, a search box, those are all affordances for humans, but they may not be really clear for machines. So, when you start working with machines, we start to do some things where we design a different kind of media type or a different kind of message model that’s focused more on machines than humans. We definitely use them, we use them every day.

One of the things I have discovered over the last years is whatever information I don’t put inside a response or a message I may have to put in source code and actually have to code into clients. The more information I can include in the message, the less code and the more opportunity for that thing to change over time, so I think what we are going to find are more and more affordance based message design over the next couple of years. We’ve had this thing called media type registration and in the last two years we’ve had more media type registrations that have in-message affordances in the last two years than we’ve had in the previous ten. So I think a lot of people are hitting on this idea that affordances are really important.


5. The first word that comes to mind to describe OAuth 2.0 is complex. In terms of API security what are your thoughts regarding this?

OAuth2 is definitely more complex than OAuth1 or 1.0.8, but I think we definitely need this kind of authentication modeling. The big advantage of OAuth is this notion of that two parties can actually share information when neither one of them holds the actual credentials for the user in case: we have a three legged patterning. So we definitely need that kind of stuff. I know the original designer of OAuth, when we worked through the OAuth2 spec got really frustrated with the process, but I think it’s leading in the right direction and while there is some built in complexity early on and some of it might have been avoidable, I think it’s still the best option for us in a widely distributed system.

One of the things we need to do in all kinds of distributed systems is reduce the dependence on single hubs or single points, and I think OAuth does a really good job of creating a model for distributed authentication. I think it’s the best model that we have and it’s the one that we need to continue to use. You can definitely get tool sets or tool kits or proxy servers to start to handle most of that complexity today; you can hide a lot of that complexity right now even while we work through the details. One of the great things about specs, through the IETF, this is an RFC, is we can make some changes over time that aren’t going to upset the apple cart. So, we’re advising all our customers to definitely consider OAuth, especially if you’re in any kind of distributed space.


6. What about web single sign-on? Do you think there’s one approach that might be emerging to be the dominant leader in this space?

I am not sure if there is one approach that is emerging, but what I can tell you from our work with customers is that we’re getting more and more requests about it. So, OAuth is this notion that I can bring my credentials to the table, so I can show up, I can walk up to your window at your web server and provide my credentials from some other party; in other words, I don’t have to actually have credentials with you. So, I think that what’s going to happen is that some third parties are going to become leaders as trusted sources in that space. So, once we have that pattern where there are some trusted sources, then lots and lots of services can say “we’ll get a relationship with that trusted source, you can walk up to the door and you have the same sign on”.

So, the next step is once you signed on here can I use the same sign on in multiple places in the same session. I don’t see any particular one but I know for example is a company named Okta which is very big in the enterprise space on this and this is good when you have all those other services sort of behind a single firewall or a single pattern. I think that is probably the first step that we need to get really good out there, there are some other companies that I can’t really think of right now, but I think eventually we need to even get beyond that. So I need to be able to carry my credentials around with me in a very distributed way, just as we make a lot of complexity of OAuth hidden, I think we will make a lot of complexity of multi sign-on or multi service use hidden as well.


7. Some failures of SOA transformation projects were attributed to excessive governance and process. Do we need governance for APIs? If we do need governance, what would be the right way to approach it?

The first part of that answer is yes, we definitely need governance. Any time we get anything going, you’re going to need some kind of governance, some kind of control, some kind of oversight. I think probably what we claim is sort of the failure of SOA modeling is not just one thing, it’s a lot of things. I think the real key is it goes back to contracts. It’s about deciding what kinds of things you want to make contractual, what kinds of things you want to govern carefully, what kind of things you want to govern loosely.

One of the powerful things about the web is that we have a lot of loose governance, we have a lot of very general things, the protocol, some media types, a couple of authentication formats and really that’s it. After that you’re open to doing what you want, anything you do adds to that and anything that adds to that limits things in the future. So, the longer you run things, the more extensive they are, the more important it is to have some kind of loose coupling.

I think a lot of times when people would try to implement SOA, they’d use a lot of tight coupling, they’d use a lot of tight mining. If you think about it in language terms they’d have a lot of strong typing rather than loose typing and we can see a pattern still today, there’s still a pendulum that’s using a lot of loose typing in languages. I think what we need to do is think about what are the things we need to govern carefully which is usually access control lists, identities; what are the things we don’t need to govern quite so carefully and that would be the actual operations, the function names. So it’s when you apply good credentialing, like OAuth, as a start, and then we apply that to a general guideline for a set of functions, I think it’s going to work out just fine.


8. What is a management API and in your opinion, what are its pillars? As an API provider, do I necessarily need a platform for managing my API?

A8 I don’t know if you need a platform, but there are definitely some pillars, especially if you are going to share your API outside, so it goes right along what we just talked about: I need access control, I need to know who’s using my API, I need to know what apps they are building with it, I need to know what kind of traffic they’re bringing to the table, what kind of errors they might be bringing to the table, what kind of bandwidth they are using.

So there’s lots and lots of pieces. When we talk about API management, we talk about a couple of things. We talk about this notion of identifying the service: we use API keys for that. We talk about authenticating the user who’s using the service: so that’s authentication of some form or type, it could be OAuth on the outside, or lots of other ones. We talk about authorization: now that I know who you are, what are you actually authorized to do. We talk about the idea of encryption: do we use transport level encryption, do we have field double encryption. Sometimes we even talk about the notion of non-repudiation: do I have to actually make sure that we’ve validated you to sign this somewhere, we validate this action. So, those are all pillars of managing API interactions.

Then you still have life cycle issues. If I make a commitment to an API, I make a commitment for a certain amount of time, if I am going to deprecate it I have to do it in a certain way, if I am going to make changes to it I have to make sure that the changes don’t break clients, especially in a distributed system/ If I invest a couple of million dollars in building a client for your API and you change that every six months that is going to cost me a lot of money. So, that’s what API management is about. It’s managing all those interactions and then managing that complete life cycle pattern. So a lot of things that we learn from SOA we can apply to this API world which means life cycle is still very, very important.


9. You mentioned versioning as one if the issues. What are some other unique challenges of rolling out and operationalizing APIs as compared to web applications or even an enterprise software?

That does lead right into it, that’s part of server management system. There are a couple of things that we talk about, patterns that we see in successful organizations. One of them is great APIs usually tend to be a very thin veneer, they’re a true interface into a set of components. Often, an organization will have lots of interfaces to the same set of stable components. Those interfaces may change relatively soon over time, but the components do not, those components very often stay very stable year after year after year.

This notion of having stable components, a little bit of scripted interface and a very thin API veneer is really important for creating a really solid implementation. So that means that I can make minor changes to the API without having to percolate Q&A all through the system, means I have a lot of agility on top, I have a lot of stability on the bottom, so that’s a really fundamental part of creating a great implementation for interfaces. And then the other is the one we talked about before which is versioning or life cycle pattern, leading clients to commit to things that aren’t going to change over time.

So our initial guidance to most people is don’t version at all. What version of the web are you using, what version of browser do you use today, what version was the page of HTML you just got? A lot of times we don’t know, we only worry if they break. So, if you can make changes without breaking, you don’t need versioning. This whole concept of implementation details as well as that external guarantee, those are the big issues in creating a functional API library.


10. So, Mike, UDDI was a failure, but repositories made service discovery possible within the enterprise. With almost 9,000 web APIs out there what are some of the channels for API discovery?

That’s actually an excellent question because I think that is our next big challenge. You talk about the 9,000 APIs - we have 9,000 snowflakes - it’s not just 9,000 APIs, it’s over 100 APIs just for shopping and in the bigger sense of things I don’t think we need 100 APIs for shopping. That means I don’t think we need 9,000 APIs but we probably need hundreds and hundreds of them. So, one of the processes I think we need to get used to is the notion that every time I create a service I don’t necessarily have to do it totally uniquely. I probably can learn from other people, I can probably stand on the shoulders of giants. There’s probably a handful, maybe just a few shopping APIs that we really need, so part of it is simplifying that case.

The other part is that even though the notion of discovery system, the UDDI pattern, didn’t really take off, the notion is really solid both inside the enterprise and outside the enterprise. So, this goes back to the API management tool. A good management interface would make it possible for me to search for services, or search for APIs within my collection, so whether you have an enterprise level management tool or whether we have something that faces outward, those are definite possibilities. It turns out there are a bunch of people working right now on an idea of normalizing how to describe these interfaces in a very generic way. There is a project called the JSON Home document through the IETF Mark Nottingham is working on; there is a process that Leonard Richardson and myself are working on called ALPS, Application Level Profile Semantics; there is one Erik Viel is working on, there is one a guy named Kevin Swiber is working on, all of a sudden we have a whole bunch of people working on this notion of describing the abstract part of APIs and I suspect, it’s probably not going to happen in the next two years, maybe it’s the next three, four or five, we’re going to end up with a generic pattern that works both on the wide distributed web and can be brought into the enterprise for describing APIs in a general way and searching for them and actually using them to generate some code very similar to the way that WSDL has done this. So I think that what we might find over the course of these ten to fifteen years is sort coming right back around and discovering things we really liked about the OAuth model in general but applying it to a much bigger picture in a successful way.


11. You’ve been presenting and writing quite a bit about hypermedia APIs; what are they and what kind of reaction has the idea been getting from the community?

Most of the things I have been talking about on the conference track has been about this notion of hypermedia; I still talk a lot about the use of hypermedia. The concept goes a long way back, more than 15 years old, it goes back to the Manhattan project, it goes back to these ideas how people follow links. The whole creation of the mouse and the pointer device was because of hypermedia, the whole creation of the web was because of hypermedia, this whole notion of the REST pattern is all based on hypermedia as a fundamental part of it. So there are lots of examples over several generations of doing this but we don’t see a lot of it in real life so it fascinated me over the last several years. As a result, I started working with some very smart people that gave me some really great feedback and I’ve discovered there are things we can do today that make it possible to use this as sort of our contract model.

Most of the talks I do today talk about that, as a matter of fact I did a couple of workshops here at QCon that both had hypermedia features to them: one was based on an article series that I am doing with InfoQ about how to implement hypermedia servers and clients. It turns out clients are really the hard part right now, and the other one was based on a book the Leonard Richardson and I are going to release in the summer called RESTful web APIs, and that takes a much broader view, it’s not just hypermedia, but it actually lists lots and lots of media types, lots and lots of standards.

So the reaction we are getting from folks, just this week, people are saying “ah, I think I see the value, I think I see this pattern, I think I see where this is leading”. In Layer 7 Ronnie and Alex and I are working with companies that are adapting this hypermedia pattern internally. Now it turns out this hypermedia model is a little more abstract, there aren’t good tools and there’s not a lot of public guidance on it, that means it’s not really a good first day strategy for a public API. The CRUD model is a much more accessible and usable strategy, but internally in enterprises I am seeing more and more interest, that’s usually a community that is a little more captive, a little bit more focused in a particular domain space and we are finding more and more people ask us about how to bring this to the enterprise, how to bring this to the table and I think the other big driver is going to be mobile. So in mobile when I have to keep downloading an app every time I want to make a change that becomes rather expensive, if I can make that change without a download, without waiting through the approval process, I think that people are going to use a lot more of hypermedia.


12. Mike, is there any general advice you’d give to developers or even organizations looking to build out their API strategy?

I think the biggest advice I would give to folks is focus on the message that travels on the network, not on the client or the server. It doesn’t matter whether you are doing this in Ruby or Erlang or JavaScript or C# or Java. What matters is the message in between because what is really happening now is we need to distribute this in a very wide way.

I think the other big advice that we are finding valuable is think about distance. We often think about distance in terms of I’m here and someone is across the world, how are they going to use this API without ever meeting me? How are they going to be able to understand it? But I think what is even more important is the distance in time: I’m going to put this API up, what happens a year from now, two year from now, five years from now, can somebody still use this API?

I think this idea of distance and this notion of focus on the message rather than the end points are really the most powerful things you can start to think about and start to design systems that work in the distance of both time and space and work no matter what kind of language is on either end of it. Now we have an API that is distributable, that’s really open, that people can use in lots and lots of ways, I think that’s sort of the key.

Donald Norman has this really great idea that a well-designed object is something that allows people to use it in ways the original designer had never intended and I think these are the kind of APIs that we want to build: things that allow people to do new things that we haven’t even thought of yet.

Nitin: Mike, on behalf of InfoQ, thank you so much for your time today.

Thank you very much.

Aug 31, 2013