Not long ago, only programmers cared about APIs. Those days are behind us, and now companies need to manage their APIs as they would any other software they provide to customers. In this episode, Deepa Goyal discusses how to manage an API as a product, and how that impacts all members of the two development teams that interact with an API.
Key Takeaways
- When an API is your product, effectively managing that product requires a partnership between developers, product managers, and many parts of an organization.
- APIs are different from other products because the primary consumer is a developer. This results in dual software development lifecycles that interact with each other in a feedback loop.
- When done effectively, API governance is an enabler that can speed up the production development cycle, not slow it down.
- Setting clear expectations is very important for your consumers to understand the maturity stages for an API.
- Measuring the success of an API includes qualitative observations and reactions, as well as quantitative data, such as time to first “Hello, world.”
Subscribe on:
Transcript
Introduction [00:05]
Thomas Betts: Hi everyone. Today's episode features a speaker from our QCon International Software Development Conferences. The next QCon Plus is online from November 30th to December 8th. At QCon, you'll find practical inspiration and best practices on how to implement emerging software trends directly from senior software developers at early adopter companies to help you adopt the right patterns and practices. I'll be hosting the Modern APIs Track, so I hope to see you online. Learn more at qconferences.com.
Not long ago, only programmers cared about APIs. Those days are behind us, and now companies need to manage their APIs as they would any other software they provide to customers. At the recent QCon San Francisco, Deepa Goyal spoke about what API product managers need. Deepa is the Lead for Product Management Strategy at Postman, and she has previous experience working at PayPal and Twilio. She's also the author of the upcoming, API Analytics for Product Managers. Deepa, welcome to The InfoQ Podcast.
Deepa Goyal: Hi Thomas. Really excited to be here.
Why developers should consider the product manager viewpoint [00:56]
Thomas Betts: So let's just start with, The InfoQ Podcast is usually fairly technical in nature and we don't usually have product managers as guests. If the person listening to the show right now is a software developer, engineer or architect, why should they care what the product manager needs to know about APIs?
Deepa Goyal: All developers do work closely with product managers, and I think they both have their share of responsibility in making product successful. So product managers have their own set of responsibilities in terms of the value they bring to their team and the role they play in defining the requirements so that the engineers are working on the right things and the right things are being delivered to the end customers. So I think it's a partnership and I think everybody needs to understand each other to be successful.
Getting started with a new API as a product [01:43]
Thomas Betts: Let's say we're starting and creating a brand new API as opposed to a different software, specifically for APIs, where do you start that process?
Deepa Goyal: Specifically for APIs, I think the process really starts with defining the requirements. And I think that's true of any product, is really starting with what is the requirement, who is the customer, why are we building this, and being really intentional about who the audience is and understanding their needs. From a product management standpoint, it really helps narrow down the requirements, we're not trying to build a rocket ship in one go.
So the definition of where to start, what does the first iteration look like, what is an MVP? Defining the boundaries of that MVP or that sprint even, is the product manager's responsibility. So that's really where the first step is, understanding who the customer is, defining the requirements, and then we get into the design phase where software architects and engineers can jump in and help design the right solution.
Thomas Betts: And so you want to start with a customer. Does that start with doing user research and understanding who your customers are and what they're looking for?
Deepa Goyal: Absolutely. I think we have to start with the user. And from a big picture strategy standpoint, it's also important to understand who the target segment is. It could be a small set of individual developers or are we building it for SMBs? Are we building for enterprise? Different solutions can work for some customers and not for others. So it's very important to understand what segment we're going after, what are their requirements, what are their pain points so that we have a clear understanding of what's the problem that needs to be solved that our architects and engineers can review and propose the right solution for.
Thomas Betts: And just to play acronym police, did you say SMB? Is that small and medium businesses?
Deepa Goyal: Yes, that's right.
Thomas Betts: So that's like a partner API, saying, that you're releasing as opposed to a fully public API?
Deepa Goyal: No. Partner APIs are different, and SMBs are more small to medium businesses that could be essentially not as large as a enterprise. It could be mom and pop shops that are trying to integrate with APIs. We saw a lot of those at PayPal and Twilio. A lot of times these are customers who want less complex solutions, they want more copy, paste code or they want simple tooling because they don't have a lot of developers. So it's really small developer teams or they want low-code and no-code tools, so their requirements tend to be a little different because they're small.
Different approaches for different API contexts and audiences [04:10]
Thomas Betts: Let's talk more about that because I know at your presentation, you talked about the different approaches, whether it's a public API, a private API, whether it's for partners for the small medium businesses, low-code, no-code options, an internal API. And those are all different flavors. API is one of those terms that really varies on what the context is. What are those different approaches that you look at based on what your target audience is?
Deepa Goyal: All of these that you mentioned, when you think about internal APIs, our users are developers within the organization. And within the organizations, it's very easy for two engineers from different teams to talk to each other and understand what is needed, establish a timeline that works for each of them. So I think in a way our internal developer, who is our customer when we're building an internal API, the customer is very approachable, very engaged and you have a lot of access to the customer. But as we get to partner API for example, the communication is managed through maybe an account manager or a sales team and you have comparatively less access to the customer, and there is an external set of developers who are trying to integrate with these APIs, and then trying to understand what their need is is a different process and their needs could vary based on their size.
And then when we go to public API, you're trying to build something that anybody can pick up and use and hopefully not require as much hand holding. So trying to design the APIs to be more intuitive and learnable and trying to understand that when we put out public APIs, the users could vary in size. It could be a single developer or like I mentioned, an SMB or an enterprise developer. So because you have this wider set of audience with public APIs, because anybody can pick it up and use them, then you have to build with that in mind to make the design more intuitive and think about a lot of things in terms of developer experience and how these APIs are presented.
Managing two groups of developers: Producers and consumers [06:08]
Thomas Betts: I can see that developer experience varying quite a bit based on those different flows. And developer experience, just like the group of developers, has two meanings. You have the developers producing the APIs, you also have the developers consuming the APIs. How do you manage those two? I'm working for the developers, I'm dealing with developer experience, how do you quantify which developers you're talking about?
Deepa Goyal: That's a very interesting aspect of working on APIs as products. Is, on your team, there are developers who are building APIs and your customer is also a developer who is integrating with those APIs, but I think their goals are very different. The teams who are building APIs are solving for making these APIs designed to be more consistent and reliable and scalable. And the external developer or the customer persona of the developer, their goal is to integrate quickly. So they are not really as involved in the infrastructure thought process of the APIs. They're on the receiving end of, they're just trying to understand how it works, make it work and move on.
And a lot of times, this is not the only API they're working with. They're probably integrating with a number of different APIs throughout their application for different use cases. Maybe they're integrating with maps somewhere and payments somewhere else, so their goal is really about integrating quickly. And the things that they think about in terms of their decision making or evaluating APIs are different versus internal developers, are thinking about the infrastructure and scalability and reliability. So I think those are two very different personas that we have to constantly think about.
A feedback loop with two software development lifecycles [07:50]
Thomas Betts: And I think that gets into the whole idea of the software development life cycle, that you actually have two software development life cycles going on constantly and they create a feedback loop that you're developing stuff internally as a producer. And normally when you're talking about the software development life cycle, it goes through design, develop, release and come back and we do small iterations. But there's also the consumer side where you're picking it up. How do those two feedback loops correlate with each other?
Deepa Goyal: I think of it on very similar lines. For APIs, we have the API producer who is going through the process of defining, designing, develop tests, secure, making sure we have observability and then we are ready to distribute the APIs. I think that's where your sales, marketing or developer relations teams come in and make the API available to the consumers and drive the discoverability. And for the consumer, they discover an API and they go into evaluation, integration, testing, observability.
The interesting thing there is, throughout the process of the consumer journey, they have to have enough feedback loops in terms of either support channels. Could be different support channels, could be ability to have enough developer community around your API so that if they have any questions, they know that there's a community that's going to help them. And if it's through support channels or forums or even things like Stack Overflow where developers can go and post their questions, then those are great feedback loops for us to learn in terms of where our customers get stuck, what are the pain points in trying to integrate with our APIs. And that becomes the input for our next iteration so that over time, we continue this loop of producing and consumers keep consuming and we can keep learning from them so that our APIs are essentially iteratively improving over time.
Customer feedback when your product is an API [09:46]
Thomas Betts: Especially in larger software products, that you'll have a customer relations team and online forums maybe in places where they can submit like, "If you want to provide feedback, here's how you submit feedback." With the APIs, because your customers are developers, you have to go to where they are. Like you said, having something show up on Stack Overflow seems really useful once they get to that integration and development phase. You've sold them on it, they want to try out your product, can they get it to do what they need it to do? And when they run into a problem, can they solve it? Is that where API's, again, they're software but they're different than other software because you're talking to developers and you have to go where they are?
Deepa Goyal: There's quite a few interesting aspects of APIs being unique and very different from consumer products or other SaaS products, is because ultimately, developers who are integrating with our APIs also do understand how APIs are built. And because both sides of producer and consumer are fairly well aware of APIs themselves, I think the consumer side developers are also very engaged set of customers. So I've had great conversations with our customers where they have a lot of ideas on how our APIs could be better, and it's interesting to see how their decision making happens.
So for example, when a developer is evaluating between multiple API offerings, they have a different idea of how will they decide on which one they want to integrate with. If let's say somebody's trying to evaluate between, "Should I go with Stripe or should I go with PayPal," then their criteria is very interesting because a lot of times I've even heard things like, "Where do you first search? Is there a community around this API," is actually something very important to them...
And they would even search, not just like, "Are there enough YouTube videos around it? Is there a Udemy tutorial on this?" Or when people ask questions on Stack Overflow, do they get answered? Those things are actually important to developers in terms of making a decision in terms of using a set of APIs or not.
Thomas Betts: I think architects are always familiar with trade off decisions, so you have to evaluate. So if you're choosing between two products, like, "This seems like it does what we need it to do, but is it going to be around next year? Are enough people using it?" And so sometimes, whether it's a better product or not falls down to it's more adopted in the community. I know plenty of JavaScript frameworks to name a segment, have been adopted simply because they're a big name.
Viewpoints to consider for API decision making [12:09]
Thomas Betts: You've mentioned a couple different stakeholders, and again, architects like to understand different viewpoints and consider all those when they're making decisions. Who are the different viewpoints that we need to consider and where do they show up in that life cycle? When do we start talking to those people?
Deepa Goyal: There's a lot of different stakeholders. Starting with definition, for example. If we think about a product management might be the primary owner of defining the requirements, but they have to work with UX research or marketing research teams to understand the right audience and drive enough research data. Then as we get into design, you have to involve the software architects and they might be the people driving the architecture decisions, or the engineering leads might be the ones driving the design decision of the API, but PMs still have to collaborate and provide inputs.
And again, that could be one person, that could be a set of people depending on the size of the organization. And same goes for development and testing, where could be the developers, be the QA team. And then we get into things like security review for example, a lot of times those are separate security teams and they would be the stakeholders who are responsible for signing off on, "Does this API meet our enterprise security standards?" And more and more companies are becoming thoughtful about their security strategy around APIs as incidences around security go up.
A lot of organizations want to build APIs, but I think it's only when they feel like they're secure enough and they're able to meet security criteria that they feel comfortable. So security is becoming more and more top of mind. And as we go into observability, there is data engineering that needs to get involved in terms of piping the right data, building the right dashboards, or it could be your engineering team. So multiple teams are involved at different stages. And then of course getting APIs out there in the hands of customers requires multiple teams again, in terms of developer relations or marketing and sales.
Developer relations generally addresses more of the broader developer community, marketing might be across the board, and sales teams are usually very focused on enterprise customers. So you have different stakeholders for different aspects of everything, and most times in large organization that ends up being teams of many people rather than single person.
Challenges when marketing an API [14:33]
Thomas Betts: I think the integration with people that are not just in the development side, you have to consider, just like if you're trying to sell a software product, your API needs to be out there, people have to find it. And so developer relations sounds like a thing that's specific to APIs. You probably don't have as much developer relation if you aren't providing an API with other software. And marketing also seems to have various roles. There's company marketing, but then there's product marketing. And so does product marketing get involved and say, "This API is a product, we need to market it specifically to different people"?
Deepa Goyal: Increasingly so. I think more and more companies are realizing that APIs are their standalone product. And when you try to position API as a product, marketing teams have a different challenge on their hands to sell this product that is not really a UI based product. I think it's a unique challenge and something that a lot of marketing folks are also thinking about a lot in the industry, is how do you do sales for a developer focused product that doesn't necessarily have a UI? And what does that look like in terms of establishing a customer funnel, for example, which is so foundational for a marketing person. It's like, "What's my conversion rate? How do you measure conversion rate for an API?" So I think there's a lot of nuance to how it's implemented and how it's done and measured.
A lot of interesting challenges there that are being talked about in the industry and people are doing in various different ways. And marketing has their broader set of responsibilities versus sales has their unique challenge of understanding different architectures that different organizations have because tooling itself is so vast. Especially at Postman, we see when we talk to a lot of customers that there is these permutations and combinations of tools that people are using and trying to understand which tool will be right for you or how this tool will fit in your infrastructure and tooling. Tooling setup is also a very interesting challenge for sales teams when they think about selling technical products.
Thomas Betts: You have to really always understand your audience, know what they're trying to solve. And that's where the marketing comes into play, that you're probably trying to convince the developer to try and use it, but the developer might not be the one making the purchase decision. If you're selling this as a paid product, you have to convince the people higher up that yes, it's going to allow better integration, it's going to make their job a lot easier, it could make their developers more productive. Those are the types of messagings that I often hear when I see an ad for API.
Deepa Goyal: There's also different personas on the consumer side. So for example, a lot of times, and exactly as you mentioned, developer may not be the primary decision maker... For example, even things like security teams might be the ones who have to review and sign off on selecting a particular API solution for their organization. So it could be a security review, it could be from a business standpoint. There are aspects like pricing that are very crucial to the decision making to the consumers, that might not be something that developers are owners for. So there are multiple personas even on the consumer side who are involved in the decision making.
The importance of API governance [17:38]
Thomas Betts: And then when does governance come into play and how important is that for product management of an API? What does it mean for the developers that are building it and that are consuming it to say, "I have to fit into this governance model before we can actually release the product"?
Deepa Goyal: I like to think of governance as driving directly from the life cycle. So as we identify different owners for every step of the life cycle, then those owners are responsible for defining the standards and best practices and policies around each of those phases of producing the API. And together, that process is your API governance framework that should ultimately enable developers to build more effective APIs and be more productive in terms of having these guidance on how APIs need to be built in our organizations.
So I think API governance is something that comes in across the entire life cycle of an API, and the developers have to be aware and be enabled with the information of what is expected in terms of how your APIs are being built so that you can meet those standards of the API governance. The process needs to be clearly communicated, documented, and we have to have enough SLAs established between all these different stakeholders so that the API governance process is not ultimately slowing people down, but enabling them. I think that's where the real challenge with API governance is. It should be an enabler, but a lot of times process can seem very imposing. And if it is not set up correctly, then it can slow things down instead of speeding things up.
Thomas Betts: I think that goes back to the point earlier about developer experience, that for the developers who have to produce the APIs, if you have the right tools in place and other ways to check that you've met the guidelines, it's not onerous to follow the rules. It's easier to fall into the pit of success, that you naturally create an API that meets all of the standard nomenclature. And on the consumer side, the developer experience is, "I can't tell that there were 20 different developers working on this because the API has one voice that it all makes sense, and it all looks like it came from the company as opposed to this development team and that development team."
Deepa Goyal: That's where the product really comes in, is the API governance strategy should enable many, many teams to build consistent and coherent APIs. So you want to make sure that you're enabling many teams to operate independently, but ultimately the outcome looks like a coherent product. Because your end customer should not really have to figure out your org structure to be able to work with your product.
Managing API distribution and sales [20:17]
Thomas Betts: That segues nicely into, I remember one of your slides was about the maturity based distribution model and working through the different phases. Can you talk about that, of how selling goes through different phases? And again, it changes for private versus partner or public APIs as well?
Deepa Goyal: Yeah, absolutely. So I think the way we rollout APIs, one of the biggest challenges with APIs is once you put something out there, people have built on top of it, it's not easy to change that. So it's very important to emphasize the iterative development instead of just rolling out big changes or breaking changes. No customer would really appreciate breaking changes for APIs, and I think they're really difficult to manage in terms of, no matter how much notice we give customers, it's always going to be a bad experience.
So to avoid that, I recommend using a framework for iterative incremental rollout of APIs. Where I recommend that irrespective what kind of API you're building, if it's internal, partner or public, you should first start to do an internal release where you have stakeholders inside the company. It could be, if you're building just for a few teams within the organization, you can get them to test it out before you actually officially make those APIs available and do some hackathons, get some feedback. So use that product thinking of getting user feedback, doing some usability testing internally and getting to a more confident state before finalizing your APIs.
That should be done for all kinds of APIs. And from there, if you are building something for partner customers then you should think about doing maybe a beta test, like a private beta with that partner and getting them to test it out, give you feedback, do some iterations, and then roll it out as a generally available partner API. And then continue on that path if you want to make them public, where you might do a public beta where you can recruit more customers to test it out and then go from there towards general availability. So I think incremental release gives you enough feedback loops throughout the process so that you're building confidently.
Define your API maturity models [22:29]
And in all of this process, I use these terms like internal release and private beta and public beta. These are all maturity stages for an API that really help the customer approach the APIs and understand where the APIs are in terms of their maturity, because setting expectations is very important. So if as a customer I know that this is a beta product, then I expect that there could be some changes being made to it. And you should actually publish that on your developer portal, as like, "This is what we mean by a private beta, this is what we mean by a public beta and this is how you can participate." Or, "This is what you can expect in a general availability product."
Setting those expectations with the customers, setting the SLAs. Even in terms of beta products, may not have as many video tutorials for example, and maybe you only build those for general availability products. So setting those expectations of customers really helps combine the developer experience expectations and the maturity model, tying everything together so it's very coherent for the customer.
Thomas Betts: And is the progression between those maturity points, is that something we can automate during our development process or does it usually require a human in the loop to say, like you said, "release this to the development teams", obviously you're waiting for feedback there. Are there baby steps we can do to say, "Have we gotten ready for the private release, the private beta yet"?
Deepa Goyal: Well, I think it depends on different teams. I've seen it be done in various different ways. I think tooling is a little bit of a heavy lift to make it automated. I've seen that at various organizations where they have internal tooling that is robust enough to quantify what kind of tests an API meets and what kind of standards so that it can automatically assign a maturity level. But I think for most places, it's generally people who are reviewing the API governance body that is reviewing and signing off to say that, "This meets our criteria." Or we can say that if an API meets a certain number of criteria, then it is ready for certain audience.
How to best support an API product and the development teams [24:31]
Thomas Betts: One thing I remember you talking about at QCon was the components of the API experience. We've talked several times about developer experience, we were just talking about API lifecycle and maturity, and the last one is support operations. Again, I think of support internally as, we don't throw it over the wall, we've got DevOps or whatever. But there is a support component. Is that also doubled when we were talking about the support for our product, but also how do our consuming developers support their product that depends on our API? And what is your role as the product management team to help them be able to support the products that they build with your APIs?
Deepa Goyal: That's very interesting. There's so many aspects of support, or the word, "support", applies in so many different places. When we think about support from a product standpoint, it's the support channels that serve the consumer and that would be being able to create a support ticket and get help when they're trying to integrate with the APIs, and these are consumers. So the consumer developer or consumer persona being able to create a support ticket, being able to get help, having SLAs in terms of it could be, let's say our turnaround time that we come into a beta product might be seven days versus for a general availability product might be 24 hours, and that really helps them understand that, what kind of support they can expect.
And as a product manager, I'm generally very interested in what is the ratio of active customers to the customers who are running into some kind of support tickets. So if there's 400 customers, if there's 3 customers who are creating some kind of support ticket, that means something very different than if it were 20. So what is that ratio, is very interesting to me and very important in terms of how effectively self-serve the product is, being able to measure self-serveability of the product. And also being able to deep dive into that data to understand what is the nature of tickets that's being created.
A lot of tools these days, like Zendesk and others, provide you a lot of ways to tag tickets coming in and categorize them and you could do some level of clustering and data mining to understand what is the nature of issues that customers are facing. And as a product manager, that is a great insight into real customers having real issues that we should be thinking of in terms of the next set of features that we build or bugs that we address.
Collaboration with developers producing and consuming an API [26:54]
Thomas Betts: One of the tenets of the Agile Manifesto is customer collaboration over contract negotiation. When your product is an API and an API is a contract, how do you collaborate with your customers effectively? Because you're defining a contract, it's almost at odds. And how do you promote a fast feedback loop so you can get the changes you need and the collaboration you need?
Deepa Goyal: It's definitely a push and pull, because I think also varies in terms of the size of the audience. I think it's easier to interact more when you have a small set of audience and you can build with them in a more collaborative way versus as your audience grows, the feedback loops are slower. And that's where having a more conservative approach in terms of the design of the API is very important. And reducing scope is really important because if we try to build a lot of things up front, then chances are that customers would run into issues and at that point, it would be too late to make changes.
I think the only real way to address that rigidity is by being more iterative in our approach of development and making sure that we have enough signal from our customers in terms of feedback loops, maybe a small sample size, but some set of customers who are always engaged and are able to give feedback as the development happens. So that hopefully, we're not designing to the point where it cannot be changed and we put ourselves into a fix. But even if we do, which totally happens in case of APIs, that's why we end up creating new versions of APIs. And then we have to think about how do we deprecate the old version and how do we get people transition to the new version, because we do inevitably run into situations where we cannot work with the existing design anymore.
Thomas Betts: Plug for a future podcast episode where I'm going to be interviewing Brandon Byers about how to develop APIs without versioning. More on that in coming weeks.
Measuring API success quantitatively [28:51]
Thomas Betts: I wanted to wrap up. We were talking about customer success and customer delight, which are really qualitative phrases that can sometimes have ambiguous meaning, depending on who you ask. What are some of the quantitative ways that we can measure the success of an API?
Deepa Goyal: For sure, there's quite a few. I think the way I like to structure my thinking around metrics for APIs is really starting with the API itself in terms of the infrastructure. Because ultimately if your API doesn't work, then any experience you built on top of it is pointless. So first of all, there is the set of metrics like uptime, error rate and max latency for example. And these are usually owned by the engineering teams. They would set these up and I think product managers have to be aware of these, even if they're not the ones owning it. I think it's important for them to be educated on them.
And once you have those in a good position, that's where we start to think about measuring the API experience that is built on top of these APIs in terms of measuring the customer journey of discoverability evaluation. Where customers are essentially discovering your APIs through blogs or social media or your developer portal, and you can measure things like views and session duration or bounce rates and time spent on the site. Or if you have in-house tooling, how people are engaging with those tools and measuring those. And then we get into usage metrics where we think about who are our biggest customers, the top customers by usage or recurring usage is a very important one.
I think in terms of the customer overall, we also think about things like time to first Hello World, is a very key metric. Where, how long does it take a developer from discovering your API to making their first API call. First API call could be something insignificant, so a lot of times we also look at time to first business transaction. So that's where for example, for a product like PayPal, that's the first actual financial transaction which could take a few API calls and not necessarily just one. So time to first API call time to first business transaction. And then we also establish things like value, which could mean different things in terms of different businesses.
So for example, if in a finance setting or even, let's take example of Twilio where you have complex customer support flows that go through various API calls, then a certain volume could establish as value to the customers. If a customer is able to make 10 calls or 10 flows every day, then it's driving value for them potentially. So we analyze the data and we establish what symbolizes or signifies value for the customer and then we measure time to value for the consumer. So there's a lot of different metrics that we can measure from different lenses.
Deepa’s new book: API Analytics for Product Managers [31:40]
Thomas Betts: You have a book coming out next month, what is that? It's API Analytics for Product Managers. Is that the right title?
Deepa Goyal: Yes, that's the right title.
Thomas Betts: What's it about and who's it for?
Deepa Goyal: My background is in data science, and I've always been very interested in using data to make product decisions. And in the API space, I've seen that there's a lot of metrics that we could be using. Not a lot is very easy to instrument or evaluate or measure. That's a pain point that's very close to my heart having been an API product manager.
So from my journey, from my learnings, I've put together this material, this book where I talk about the various aspects of API product management. I also describe API lifecycle, consumer lifecycle, doing user research. But ultimately a lot of focus on metrics, in terms of what are the different metrics that you should be measuring and thinking about to make the best decisions in terms of building a great API experience and driving a great API business.
Thomas Betts: Well, I think that's a great end of the conversation. That covers pretty much everything API product managers need to know and some of what engineers and architects can also benefit from knowing about. I want to thank again, Deepa Goyal for speaking at QCon and being my guest today.
Deepa Goyal Thank you so much, Thomas. That was a great conversation. Really honored to be here.
Thomas Betts: And I hope you'll join us again for another episode of The InfoQ Podcast.