BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles A Standardized, Specification-Driven API Lifecycle

A Standardized, Specification-Driven API Lifecycle

Bookmarks

Key Takeaways

  • There is not a single API specification to rule them all, they all work together.
  • API specifications power more than just documentation as part of the lifecycle.
  • API design-first is not the only way to produce a usable API specification.
  • API specifications are essential to address regulatory requirements.
  • Teams are more collaborative and coordinate better using API specifications.

At QCon Plus last November, Kin Lane, Chief Evangelist with Postman and the Open Technologies Team lead, presented on API specifications. API specifications are essential to Lane and Postman. So he wanted to share a bit of how they see API specifications impacting how they produce and consume APIs.

API Specifications Are Not New

API specifications are nothing new. Web Service Description Language (WSDL) has been defining web services for the last 20 years. As soon as the more web-based approach to APIs emerged, we also saw WADL evolve to help us describe the surface area of our APIs. But it wasn’t until Swagger came out in 2011 that our documentation became much more interactive, more hands-on. Helping developers understand what an API does and how they can put it to use, we started seeing the benefits of API specifications.

By 2015, Swagger had grown. It was at version two. It was put into the Linux Foundation, creating the OpenAPI Initiative to move forward the spec, and rebranding it as OpenAPI. Also, helping us describe our HTTP APIs and webhooks for the last five or six years and providing a cornerstone of how we define, design, deliver, deploy, and deprecate our APIs. Along the way, AsyncAPI emerged as a sister specification, allowing us to use the same vocabulary but for describing the event and message-driven APIs, providing a format that we can use across multiple protocols.

OpenAPI

Let’s look at just OpenAPI. OpenAPI allows us to describe the surface area of our synchronous APIs—describe the information, what an API does, where you can find it, the servers. More importantly, each response’s individual paths, parameters, and detail, including the schemas parsed as request bodies or returned as part of each response. OpenAPI provides a machine-readable JSON or YAML way to describe what our APIs are, what HTTP/1.1 APIs do, and allows developers to use that specification at different points of the API lifecycle.

AsyncAPI

AsyncAPI allows us to describe the surface area of our asynchronous APIs, building on what OpenAPI gave us and defining the channels, the messages, and the bindings. For example, publishing and subscribing API across TCP, MQTT, AMQP allows us to determine how these APIs work across multiple protocols and describe what schemas are used as part of the payload for publishing and subscribing what messages get sent back and forth to these asynchronous APIs. Both OpenAPI and AsyncAPI allow us to describe most of the APIs in our toolbox today.

JSON Schema

JSON Schema allows us to describe and validate the payloads of these APIs. It gives us a machine-readable way to describe the objects, their properties, what type of data they contain, and then what’s actually required to validate those objects. JSON Schema is used across multiple OpenAPI, as well as AsyncAPI. They have their own vocabularies of JSON Schema that allow them to describe and validate the request and response payload in the case of OpenAPI. They publish and subscribe payloads in the context of AsyncAPI. Again, providing a machine-readable way that allows us to describe what our APIs are doing—what resources are being parsed back and forth, and what capabilities are encapsulated in these very abstract APIs that can often be hard to see and validate.

API Specifications Power the API Lifecycle

API specifications are used to power multiple stops along the API lifecycle. Most notably, it’s used for documentation. A lot of people who use Swagger believe Swagger is documentation. In fact, Swagger is a specification, and you can use it to generate documentation. Swagger, OpenAPI, and AsyncAPI are used to describe the technical details of what each API does and then allows us to publish documentation to always keep our documentation up to date. Thus, we deliver consistent documentation across teams using that API specification to generate docs using open source or commercial options.

The API specifications are also used to create mock servers. These mock servers can be part of a design-first approach to an API. First, design it and mock it so that you can make sure it does what it needs to do. Next, it can be used as part of QA and testing. Then, some groups are using it for sandboxes in production environments, so developers don’t have to play with production data when they’re learning what an API does.

API specifications are also being used as scaffolding for testing. For example, allowing API producers to generate contracts, integration, performance, and other types of tests that help them make sure an API is doing what it should do and running those tests as part of pipelines, or scheduling them using a monitor.

The same approach can be used for security. You can use your OpenAPI specifications or async to scan your APIs’ surface area, look for vulnerabilities, and audit the authentication. Next, check for other common vulnerabilities using the OWASP Top 10, trying to understand whether your APIs are as secure as they should be consistent across teams. Then, another common way that API specifications power the lifecycle is through code generation. Many companies use OpenAPI and AsyncAPI to generate client or server-side code, so you can actually use the specification to generate code that allows you to deploy the API, deploy it to a gateway, or generate SDKs in a variety of programming languages.

This allows companies to produce SDKs, libraries, and code snippets in various programming languages without having to have those programming language skills on their team. This is a critical look at why specifications matter. It’s not just for documentation; it’s for the whole lifecycle. One spec for an API can be used to generate docs, mocks, tests, secure that API, and then create the code you’re going to use to bring that API to life or put it to use in integration or an application.

Many Ways to Bring Your API Specification to Life

There are many ways you can bring your API specification to life. One of the most notable and more recent approaches is design-first. You begin with an OpenAPI or AsyncAPI, generate a template, and then hand design it, ensuring it precisely does what you need it to do. Then generate a mock server and publishing docs so that other stakeholders and other team members can use that and give feedback on that API, allowing you to design an API, mock it, and document it before you actually write the code. Once you’ve iterated upon it enough, you can publish it as an API.

The second common way for producing an API specification is code-first. Teams begin by writing code, doing what they do best. Then, while you’re writing the code, you either annotate it or use a gateway or a proxy. Next, you can generate the OpenAPI and AsyncAPI from the code. Next, you develop the OpenAPI or AsyncAPI, which can be used to create documentation tests and help secure your API.

Another common way is to reverse engineer web or mobile applications and generate the OpenAPI or AsyncAPI for APIs that are already in production andalready being used in applications. Then, you just need to have documentation published and kept up to date. Next, generate tests and secure that API, allowing existing APIs to be brought into a more standardized API lifecycle, along with the other APIs that are either design-first or code-first.

Industry Standards Use Specifications

API specifications play a crucial role in industry standards, most notably PSD2. You can find OpenAPIs for the PSD2 standard out of the European Union, which is being used to govern and regulate how financial and payment APIs are being deployed and evolved. In addition, you can see OpenAPIs being used for the FHIR specification, defining how APIs are delivered in the healthcare space. Providing all the technical details of each request and response helps make healthcare more interoperable. 

Open Referral is another API standard that uses OpenAPI to give definitions for municipal, city level, county level health, and human services, making it so that cities can share data more efficiently across states and a country. Another is OpenTravel, shaping how we deliver APIs within the travel industry, working on the airlines, hospitality activities, and other things. Again, providing a common set of standards can be used to deploy travel APIs, making the industry more interoperable.

Another example is the Open Geospatial Consortium. They’re doing geo API standards, and you can find OpenAPI specifications for their APIs. This is just a snapshot. There are many other standards, including the automobile industry and insurance. In most industries where APIs are making an impact, you can find OpenAPI and increasingly AsyncAPI to define that landscape.

Specifications Provide a Framework for Governance

When in place, OpenAPI, AsyncAPI, and JSON Schema provide us with a framework for realizing governance across large enterprise organizations. OpenAPI and AsyncAPI provide us with the ability to set design guidelines. How should our paths be described? What parameters? Should they be camel case or snake case? What should a schema look like? How do schemas evolve to minimize breaking changes? These design guidelines can be codified and applied in a machine-readable way to OpenAPI and AsyncAPI. Linting rules. It’s very common to lint the OpenAPI or AsyncAPI describing an API, so you can validate that it has a particular structure and form. That’s what linting provides us. We can define rules that make up our design guidelines and overall enterprise governance. We can apply those rules against these API contracts that we’ve defined. Then we can apply that using contract testing, so we can validate that those rules are being applied at design time, development, and build time.

These specifications allow us to get a handle on change management. These objects are defined by schemas. These APIs defined by OpenAPI and AsyncAPI providing access to those objects will change over time. OpenAPI, AsyncAPI, and JSON Schema allow us to quantify that change. Apply semantic date-based or other types of versioning to this, and handle how these resources and capabilities evolve. Then we can automate all of this. We can automate it, so it’s in the design tools, and get real-time feedback when it comes to editing OpenAPI, Async, or JSON Schema. You can apply at the pipeline or develop time in the IDE or the command line and allow governance to be realized throughout the API lifecycle and across development teams, helping them automate how governance is applied. Then learn along the way that good governance always has the necessary resources baked into it. Each governance opportunity becomes an opportunity for learning about why that governance matters and how it works.

API specifications are how we work together as a team, so you see that evidence in the documentation. This is why Swagger UI and Redoc and specification-driven documentation have become so critical. It’s how we API producers communicate with API consumers. As they’re growing and being applied across the API lifecycle, these specifications are opening up opportunities for collaboration. We can now share artifacts. I can share what I mean when I say API pagination. This is the specification for it. I can share what I mean when I say this is an image object. I can share the JSON Schema for that. Sharing and collaboration around these artifacts allow us to work closer together and have the same meaning codified in these structures. Then because we can bake this in with our existing source control, we have these artifacts. They can live alongside the code we use to deploy, automate, consume, and work with these APIs.

Our OpenAPIs and AsyncAPIs live in the workspaces where they’re being designed. They live in the GitHub, GitLab, and Bitbucket repositories where the code is being committed and used across the infrastructure to deploy and automate with those APIs. All of this allows us to standardize across teams, so we’re communicating better, collaborating, sharing, and having these artifacts that help us understand what it means when we talk about these very abstract API concepts. It helps us get on the same page, standardize how we do things, and automate that through source control. This can be done at the team level, the organizational level and using standards at the industry level, allowing APIs to be more consistent and iterative throughout their lifecycle.

Specifications Are the Language of Our Business Operations

These OpenAPIs, when they mature, provide us with the ability to define the different domains of our operations. This is how we separate the different lines of business, the other domains that we use to operate. It can very much describe and define how our teams are made up—how we organize our teams and work within these different lines of businesses and groups. OpenAPI, AsyncAPI, and JSON Schema, with tagging across all of them, allow us to carve up all of our resources and digital capabilities within these different domains, describing the other resources we have—the various images, videos, people, and addresses, all the extra digital resources we have, and the capabilities, algorithms, and actions taken across those resources.

Describing all of this in OpenAPI and tagging things, helps us establish and formalize our vocabulary for how we describe this. We’re using these schemas that we’re publishing to APIs as part of requests, getting back as responses, publishing, subscribing in an event-driven API world. We’re standardizing how we describe these objects and how we talk about them. We’re reducing redundancy across these resources and capabilities, creating much more logical domains for how we operate. Make sure that it is as efficient and well-defined as possible. Then the domain experts can really help guide and manage this part of our business operations.

Specifications Are For Humans and For Machines

Specifications are for humans and for machines—even though they’re written in JSON or YAML and very much machine-readable and structured to be imported into different services and tools. They can be used as part of source control. The API lifecycle can be automated in that way. They are very much for humans as well. It’s essential that we use them, iterate upon them, document them, and make them accessible to humans for use across their work. Specifications are often seen as a developer tool. Increasingly, they’re being used by project and engineering managers, API designers, and architects to shape the domains across their enterprise organizations and used as a way of doing design reviews, security reviews, and other ways. Using it as an artifact to guide conversations with developers who build the APIs and the applications on top of them. Specifications need to be there for them to be up to date because humans depend on this. The specifications are helping us address a lot of the human challenges and friction that we encounter across our organizations. It’s important to elevate it, zoom out, and see specifications as not just being documentation or not just being code gen, or just a JSON artifact on a server somewhere. This is very much a contract that humans use to move forward our operations and find success.

A Standardized, Specification-Driven API Lifecycle

A standardized API lifecycle means that we have all these resources and capabilities defined as machine-readable artifacts. We’re not just focusing on HTTP/1.1, REST, or the existing ways of deploying APIs. We have a diverse toolbox of artifacts that allow us to describe event-driven and message-driven, GraphQL, gRPC, Kafka, and NATS. We’re able to use a full suite of API solutions that help us describe, define, deliver, and operate these APIs in a way that benefits both the producer and the consumer. This helps us standardize how we’re defining and designing APIs—how we’re delivering,how we’re doing it at scale across teams and across organizations. When you go to a large enterprise group and put an API to work, it has the same characteristics. It has the same properties parameters. It has consistent documentation. It has the code libraries I need, and they use conventions that match the programming language I’m using.

All of this is driven by these API specifications. OpenAPI and AsyncAPI are very much being used as the guardrails for this. They’re being published to a central catalog, either internally within an organization or externally, with partners and third-party consumers. These OpenAPIs, AsyncAPI, and JSON Schema can be used in any services or tools. The API producer has a much smoother, high velocity, efficient, agile, and nimble way to deliver APIs, responding to changes and getting their APIs out there. The consumers of those APIs have access to those same contracts, so they know when things are changing. They’re able to automate, build, and integrate around those APIs without wading through the documentation. They can just import an OpenAPI. They can import an AsyncAPI. They can use the JSON Schema to validate and test. Then they can grab all of the tools that an API producer has built around these specifications and put them to work.

A standardized specification-driven API lifecycle is what you’re seeing emerge within most enterprise organizations that have spent the last decade doing public APIs or the previous five or six years heavily investing in microservices. You see them working on getting a handle on using OpenAPI across their enterprise organization. First, adopting and learning how to use AsyncAPI. Then using JSON Schema to get a hold on their internal data, schema, and artifacts that are being parsed around. This is the number one priority for organizations, architects, and leadership in some of these more progressive forward-leaning companies to adopt these API specifications, standardize around them, then, apply it across a well-known API lifecycle.

Summary

Many folks are down in the weeds regarding APIs and just focus on docs or code-gen or design and the architecture around these OpenAPIs. Hopefully, this zooms out and shows how OpenAPI, AsyncAPI, and JSON Schema are working together. More importantly, how they’re applied across a consistent API lifecycle. Because once you can standardize and stabilize it across the lifecycle in this way, this is where you’re going to start seeing the benefits when it comes to operational velocity, performance, quality, and overall governance of what’s going on. I recommend heading over to OpenAPI, AsyncAPI, or JSON Schema, all three open-source solutions that let you describe how you will be doing your APIs. Each of those communities has a lot you can learn from and work as part of your operations.

About the Author

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Needs editing

    by Joe Kraft,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    This was so poorly-written it was hard to read. Bad punctuation, incomplete sentences - this should have been edited much harder.

  • Re: Needs editing

    by Roxana Bacila,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hi Joe,
    Thank you for reporting the problem. We are currently in the process of copy editing this article.
    All the best,
    Roxana Bacila
    Head of Editorial Operations
    InfoQ Enterprise Software Development Community

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT