The Compounding (Business) Value of Composable Ecosystems

Looking at the cloud native landscape, it is clear that there is a lot of business value in both cloud native and open source. Today, it encompasses companies with a combined market cap of $14.3T and funding of $29.5B. However, looking from the outside, it can be difficult to understand how something that is “free” can generate so much value. Being “free” and open source doesn’t hinder the value of these projects to businesses and end users, rather it unlocks it. The composability of open source ecosystems allows the innovation and value of the whole ecosystem to compound on itself.

Composability as Compound Interest

In the finance department, they say compounding interest is the eighth wonder of the world because your money makes money. In software, IT, and open source, composability is the first wonder of the world because it allows innovation to build on innovation. This is what we mean when we say we are standing on the shoulders of giants. 

Composable projects and products can be stacked, recombined, reused, and built upon to meet the unique needs of each customer. In cloud native, this composability allows developers to iterate and ship functionality fast, thus creating compounding value to businesses from financial services to healthcare to telecommunications and transportation. Buying into open, composable ecosystems, like cloud native, can lead to better products, larger markets, and better business outcomes for both vendors and end users because they are composable.

Composability Broken Down

Before we dive into how composability helps us achieve these business outcomes, let’s take a step back and look at what composability actually is. Composability is “a system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements.” 

For anyone that has worked with end-user companies (companies that use, but don’t sell software) before, you know that while many of the broad challenges may be the same (I need to run containers), they each bring their own quirks (but we need static egress gateways for our firewall). A composable system helps tackle these common challenges while still allowing the choice to select components that meet specific requirements. The cloud native landscape is so large for exactly this reason, end users need choice to meet their precise business needs. Now that we understand a little more about what composability is, let’s see how it applies to the real world.

Composable Interfaces

Storage

In the beginning, many parts of Kubernetes, like storage, were “in-tree” as part of the core Kubernetes code. While shipping it all as one made it easier to install, companies wanting to add support for their storage system (or even fix a bug in an existing volume plugin) had to align with the Kubernetes release process. In addition, third-party storage code caused reliability and security issues and the code was often difficult (and in some cases impossible) for Kubernetes maintainers to test and maintain.

The Container Storage Interface (CSI) was developed to standardize how storage systems expose themselves to containerized workloads and make storage composable. There are now over 110 CSI drivers for a large variety of vendors and storage systems. Creating a composable interface both increased the number of options for end users and the number of end users that could use Kubernetes resulting in a larger potential market for storage vendors. Not only that, but it also reduced the burden on Kubernetes maintainers and increased the security of core Kubernetes. Not bad for an interface.

Networking

Similarly, in networking, there is the Container Networking Interface (CNI) which “consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers.” Often in the Cilium community, I see users who have started with simple CNIs like Weave or Flannel that later need to switch to an advanced CNI, like Cilium, to meet their scalability, observability, or security needs. If they had to rewire their entire cluster including every interface, it would be almost impossible to change. Since the CNI has standardized container networking, swapping between CNIs is a much simpler task. It is even possible to do a live migration. With the CNI, end users can choose the CNI that fits their needs and switch if/when they change.

eBPF

eBPF is a technology that allows users to safely extend the functionality of the Linux kernel without waiting years for the kernel development process. By allowing users to select and add additional functionality at run time, it unlocks many new possibilities. Cloudflare uses it for DDoS mitigation, Meta uses it for high-performance load balancing, and Cilium uses it for cloud native networking, observability, and security. By moving the functionality outside the main kernel development, it drastically speeds up when users can take advantage of new features. It also unlocks new use cases, like security observability with Tetragon. eBPF has made the Linux kernel composable and brought many valuable new features, functionality, and use cases to the cloud native ecosystem.

The Value of Composable Interfaces

Making Kubernetes storage and networking composable had many benefits across the ecosystem for end users, vendors, and maintainers. First, end users have more options. This broadened both the number of companies that could use Kubernetes and the use cases it could satisfy. They can also switch if their needs or use case changes without having to start from scratch. Second, more end users means more money for vendors. By widening the user base, more users may want a support contract, enterprise features, or a managed service. Finally, by making it more modular, the projects are easier to test and maintain which reduces the burden on maintainers. Composable interfaces have benefits for people all across an ecosystem.

Composable Ecosystems

So far, we have talked about the value of being able to swap individual components and the value that provides to businesses. However, that is only one part of the equation. Composability isn’t just about what projects and products your stack is made of, it also includes the composability of the ecosystem as a whole. The value of an ecosystem is not just the sum of its parts, but rather the interrelationships between the parts and how they can be assembled to meet the needs of the ecosystem and end users.

The ideas, people, and tools that make up an ecosystem can be composable too. When that happens, the value of an ecosystem truly unlocks. Let’s walk through some real-world examples of how the composability of each of these has increased the value of the cloud native ecosystem as a whole.

Ideas

Observability, “being able to fully understand our [distributed] system,” is a hot space because it helps us find and resolve the production bug that paged us at 3am. A few years ago, there were many different ideas on how to best implement observability including OpenCensus and OpenTracing. These two projects combined their ideas and effort into OpenTelemetry which is now one of the fastest developing projects in cloud native. Rather than having two competing efforts at standardization, the two projects composed their ideas into a single project with a strong backing to move the state of the art forward. Bringing the projects together allowed the new OpenTelemetry community to select good ideas from both OpenCensus and OpenTracing and combine the contributors to move the project forward faster resulting in a better project for the ecosystem.

People

In composable ecosystems, people can swap in and out of projects to meet their interests, market gaps, and customer demands. For instance, Matt Butcher created Helm as a package manager for Kubernetes, Brigade as an event-driven scripting platform on Kubernetes, Krustlet which is a kubelet for running WASM workloads, and even helped write the Illustrated Children’s Guide to Kubernetes. Because the cloud native ecosystem is composable, he could build each of these projects on top of his existing work as he saw the need for them. Composability allows people to build upon their knowledge and end-user obstacles rather than having to start from scratch. This can create valuable projects solving end-user needs and builds new business opportunities for people in the ecosystem.

Tools

Prow is a Kubernetes-based CI/CD system that runs the test infrastructure for Kubernetes. The use of the projects has now expanded across the cloud native ecosystem to over 24 different projects and products. Reusing the same tools across an ecosystem makes it easier for developers to contribute to projects (because they are already familiar with the tooling) and it allows multiple projects to take advantage of improvements and automation in Prow. It also helps new projects get off the ground faster because they don’t need to reinvent their CI/CD. Better tooling helps projects deliver features faster, create better products, and improve the end-user experience.

The Compounding Business Value of Composability

Ecosystems are made of many different parts including ideas, interfaces, people, and tools. The web of interactions between all of them generates the most value when they are composable with each other. This ability to combine and recombine to meet unique needs of end users generates business value by:

1. Helping projects ship features and functionality faster
2. Creating better products more closely aligned with end-user needs
3. Generating new business opportunities from gaps that can be plugged into
4. Increasing the user base, use case coverage, and potential market size
5. Producing better business outcomes for both vendors and end-user companies

Composability powers the cloud native world and is what has made it so valuable for businesses too.