Transcript
Chris Richardson: Welcome to my talk on microservices platforms. Really, what the talk's about is a simple idea, taking the concept of team topologies, platforms, and platform groups, which is another term for platform teams, and using those to accelerate the delivery of applications that use the microservice architecture. Basically, it's all about reducing the cognitive load of the service teams, enabling them to deliver better software faster. I've been building software for a million years now, at least that's what it feels like. It did mean that I was actually around when patterns became popular within the software community back in 1994, 1995. I was really good, and that shaped my thinking ever since.
My brain is full of 30 years of patterns now, as well as 30 years of Java. I've just done various things. I've worked on Lisp systems, created the original Cloud Foundry. Then I also, for the past 10-plus years, been pretty much focused on the microservice architecture, just helping organizations around the world improve how they deliver software. I'm excited, so my book, "Microservices Patterns," came out like 7 years ago now, which is still quite current, remarkably. I'm in the middle of working on a second edition of the book, so one day it will come out. I just think about microservices most of the time.
Microservices Platforms: Why?
In this talk, I'm actually going to walk through six different patterns that I've identified that will help with the delivery of microservices. I'm sure there's actually many more. I want to start off just by talking about why platforms, why do they matter? What are the benefits in the context of developing microservices? I want to take a step back and talk about what software development is all about, and I like to think that at the heart of software development is a feedback loop. We in IT, we develop software, and then we put it into production, put it into the hands of the users, and we get feedback from that. It's really critical for businesses today to have short feedback loops so that they can thrive in today's volatile, crazy world, which is unpredictable from one moment to the next. Ages ago, I used to talk about brick-and-mortar businesses being disrupted by digital technology.
Since then, we've had pandemics, we've had wars, we haven't had tariffs, now we do, now we don't. It's just completely volatile. Businesses quickly need to be nimble. There's a lot of research that shows, and this is one of the findings from the keynote, that the better you are at delivering software, as defined by continuous stream of small changes, the more successful the business is likely to be. Fast flow is really important. To actually achieve fast flow, you need a combination of three things. You actually need a process, or specifically DevOps, which I'm actually not going to talk about today. That's how an organization should work. You need a properly structured organization, specifically one that follows the ideas of team topologies, which is actually a collection of patterns for structuring an organization for fast flow. Then you need an architecture that enables both of those things, which, at scale, is often the microservice architecture. I'm really focusing on the organization and the architecture piece of this in today's talk.
What is this team topologies thing? That is a collection of organizational patterns and principles for fast flow. It's basically, how do you structure your organization to deliver software rapidly, frequently, and reliably, using DevOps, actually? There are several different concepts involved. The first one is there's four different team types. Most of the work is done by what is known as a stream-aligned team. That is a team that is responsible for the end-to-end flow of work that is taking requirements and turning it into code running in production. These teams are generally small, five to nine people. Those are the teams that are doing the vast majority of the work. Then they're supported by three other team types. There's enabling teams that act as consulting teams that help teams acquire new skills. There's complicated subsystem teams that focus on domains that actually involve deep skills like math, for example.
I'm not talking about them so much in this talk. Then, lastly, there are what used to be called platform teams, and then they got renamed into platform groupings. I reject that, and I just call them platform groups in the second edition of "Team Topologies." Which is what I'm really going to talk about today. You've got these. I'll talk about what platform groups do. Then there are these three different interaction styles. Most of the interactions are via X as a Service, one team consuming the output of another in a self-service fashion. Occasionally, teams need to collaborate to discover new capabilities. Then, of course, enabling teams, which are acting as consultants, facilitate with the team that is learning. I'm going to talk about these interaction types quite a bit. Then there's a set of principles, and the most important principle is about managing a team's cognitive load.
The human brain can only deal with so much complexity. Collectively, a team, a set of brains can only deal with so much cognitive load as well. They only have a certain cognitive capacity, and you do not want to exceed that. Because if you cognitively overload a team, like just in order to get the work done, there's just so much mental effort, then that actually reduces team performance, and it ultimately impacts their mental health in terms of stress, and burnout, and so on. As Nicole mentioned in the keynote, a key part of having a great developer experience is minimizing the cognitive load of a team. As you're going to see, that is one of the key goals of having a platform or a collection of platforms. That's team topologies and cognitive load.
Then in terms of what is my definition of the microservice architecture. It's an architectural style that structures an application as a set of components or deployable units, which also go by the name of services. Then those services have two essential characteristics. They are independently deployable, meaning a service can be built, tested, and deployed in isolation from other services. They are loosely coupled, or specifically loosely design-time coupled, which means that a change to one service rarely requires other services to change in lockstep. Notice in this definition, I'm not talking about claiming that services should be small or numerous. That might be implied by the name, but that's not an important characteristic. The reason I'm talking about microservices here is because they actually enable fast flow. The way they do that is that they enable the teams that are developing in them, the stream-aligned teams, to be independent.
That comes from those two properties. Because the services are loosely design-time coupled, a team can change their service without having to coordinate that change with other teams. Then because the services are independently deployable, a team can deploy their service without any kind of collaboration with those other teams, at least the vast majority of the time, which means that teams are able to work separately most of the time, which is essential for fast flow. The challenge that you have is that the microservice architecture is quite complex. If you go look at the microservices patterns language, most of the patterns there are solutions to problems that you encounter when implementing microservices. There's a lot of different patterns that you have to implement. If you look at each pattern, it's a combination of three things. Application logic, which is what the team should be focusing on. There tends to be a lot of plumbing, so supporting infrastructure code, connecting to databases and message brokers, and logging and so on.
Then there's infrastructure services as well. Teams should be focused on application logic, and for them, having to deal with plumbing or infrastructure services would be an excessive burden. In some cases, it requires deep expertise, so it would impose a significant cognitive load. If each team was dealing with this, it would be duplicated effort. You'd end up with multiple bespoke implementations, which would be a maintenance nightmare. Then, on top of that, there are these cross-cutting application-wide concerns that don't really belong to any one team. Not only that, it's not just patterns. If you go look at the source code for a service, it's not just the application code. There's build logic, a definition of the service's dependencies, and task definitions that compile and test and package that service. There's also some deployment logic. There's also the definition of the service's deployment pipeline. Then there's probably Infrastructure as Code, like Kubernetes YAML, for example, that's there to deploy the service.
You don't want the team reinventing the wheel for all of that, because that would be an excessive burden. There's even more. Not only do you have the services, but then there's all these infrastructure services, including the infrastructure that runs the deployment pipeline, services that are global, like your message broker, for example, and then the deployment infrastructure. It would be a burden for all of the teams to actually have to take on that work themselves. That's the big motivation for platforms and platform groups. From a team topologies' perspective, a platform is an artifact that reduces the cognitive load of a stream-aligned team. It could be just a wiki page, but usually it's a tool or a library or some kind of self-service, SaaS-like solution providing a capability. It's developed by a platform group, and it's consumed as a service by the stream-aligned teams. That's the big idea here.
If you look at what a platform group is, it's actually a composite team type. It most certainly would have a stream-aligned team that develops the platform, but there might also be an enabling team that provides a consulting function that helps the stream-aligned teams be more successful with that platform. Then in terms of collaboration, this is primarily X as a Service. It's all about self-service platforms, but then the teams will collaborate to evolve the platform. Then, as I mentioned, the enabling team that's part of the platform will also consult with the service teams. The primary benefit is that it reduces the cognitive load of the teams, enabling them to work on their specific business functionality and actually deliver value. Then the platform is a single standardized implementation of some important capability. It's done right just once. Then the platform group itself can focus on developing skills in that area. Unlike the stream-aligned team, which is focused on delivering business functionality, the platform team can dive deep into whatever their particular more technical problem domain is. That's the big idea with these platforms.
The Service Foundation Platform
As I mentioned, I identified six different platforms, and there's most likely more, but I thought this was a good start. I'm going to talk about each one very briefly in turn. The first one I want to talk about is the service foundation platform that simplifies the creation and maintenance of services. As I mentioned a little while ago, if you look at a service, there's a lot of plumbing, there's build logic, there's Infrastructure as Code for deploying the service. There's a lot of stuff, and actually setting up a new service from scratch would be quite a burden on the teams, and maintaining all of that would be quite a burden as well. The solution is to have a service foundation platform that simplifies that part of service development. The service foundation platform is comprised of two parts. There's the service template, which, as the name suggests, is a template that can just be cloned to create a running service.
Then there's a service chassis that is a framework that the template is built upon. The idea of a service template, super simple. It's a complete running service that a team can just copy and drop in their business logic, and they've got this testable, deployable, observable service up and running relatively quickly. Big reduction in cognitive load there. The problem you have is that a template, and this is same as true with code generation as well, it is basically glorified copy and paste. Then what's more is the template is changing. When it's time to make an update to the plumbing for the services, you now have multiple copies of the code and it's likely to be slightly different because each service was cloned from a slightly different version of the template. There's potentially a massive maintenance task in updating all of these services. That's where the service chassis comes in.
The idea is you extract out most of the functionality that is in the template into this chassis, which is this framework. Then that framework is just referenced by the template. Hopefully, the template is really small, so the amount of copy and paste is significantly reduced. When it's time to make a change to the plumbing, you just update the framework, release a new version of it, update the service template to use that version, and then update each service to use the new version of the chassis. Fingers crossed, you've just rolled that change out and it's just a one-line version update. Of course, there are scenarios where it's a bit more complicated than that. A lot of the time, the upgrades are quite straightforward. If you have more complex upgrades that require code changes to each service, which includes updating stuff that is in the service template, one option is for the teams to do it themselves.
A much better option is for there to be some kind of update or migration script or some tool that you can apply to each service's repository to create a pull request. That might be a deterministic tool like OpenRewrite, which provides large-scale refactoring of your codebase. Or maybe, and this is my obligatory mention of GenAI, because that's the only thing that's real today, you might be able to use GenAI to actually do this update. You could actually give it the diffs of the service template, and maybe it would figure out how to apply those diffs to each and every service and create a pull request for it. Who knows, because every time you use GenAI, it's a roll of the dice as to whether you'll get something usable out of it. That's the service template, or to be more precise, the service foundation.
The Security Platform
The next platform I want to talk about is the security platform. Who here, first off, understands security and likes it. Because to me, it's like a prime example of my brain does not have the cognitive capacity to maintain knowledge of how OAuth authorization flows actually work. I learn it for a while, and then a month later, it's paged out. I don't know if that's because it's complex or because I'm getting old and my mental capacity is diminished. Security is really important, yet at the same time, it's really complicated. There's a lot of different areas to that, and I just want to talk about three areas that can impact developers. One part of security that's obviously important is authentication, verifying the user is who they claim to be, logging them in, in other words. There's a whole complex mechanism involving OAuth and OIDC and IAM services, and so on, that result in services being handed a token.
Then they have to do authorization based on that token, which might be hard-coded Java authorization rules, or perhaps they delegate to an authorization service like also cloud. There's also access control lists for things like Kafka topics and stuff. This is not what developers should be spending their time thinking about, because this stuff is hard, or at least for me, it's hard. Then, on top of that, there's also transport level security. You want to secure the communication between services as well, and that involves technologies like certifical authorities that are handing out certificates and so on. That's also quite complicated. The obvious solution is to have a security platform that takes care of that. This platform consists of two parts. There's a bunch of infrastructure services that provide the security mechanisms that I just described, like an IAM service, certificate management, which might actually be a service mesh, for example.
Maybe there's the authorization service as well. You've got that infrastructure. Then there are elements that provide security that are incorporated into the service chassis so that services are actually secure by default. You build them on top of the chassis. Then, let's just say automatically every REST endpoint will require a JWT. It knows how to get the JWT signing certificate from the IAM service. The idea is that this insulates the developer from the complexities of security, and the only security aspects they need to think about are writing the authorization rules that determine which operations can be invoked by whom. That's about it. That's the security side of things.
Infrastructure Services Platform
The next platform I want to talk about is the infrastructure services platform. Services are not these standalone things. They need a bunch of infrastructure services in order to run. Most obviously, the services need database servers, which need to be provisioned and managed. Typically, even if you're running on Kubernetes, you'll probably use AWS RDS or Aurora for the service databases. Then there's application-wide infrastructure like Apache Kafka. Maybe you use AWS MSK. On the one hand, it sounds simple, but on the other hand, setting up cloud resources on AWS and other public clouds is incredibly complicated and requires deep expertise, starting with getting an access key and taking it from there. It's like really complicated. You don't want teams dealing with this, because they should be focusing on their business logic. It makes sense to have a platform to do this. Once again, the platform consists of two parts.
One part provides the shared services like Apache Kafka and other pieces of infrastructure. Then there's what I call an infrastructure orchestrator. What that does, it enables the service team to say, my service needs a Postgres database with this capacity. Then the orchestrator is responsible for taking that service specification and creating the appropriate cloud resources, which are then managed. The teams can just go, I need this. They can express their intent and the orchestrator takes care of it. There are a few different solutions out there. One that I've used in the past in the Kubernetes world is Crossplane, which if you can get past the word soup of the documentation, it's actually really interesting. It basically extends the Kubernetes API. Let's say the platform team create a new resource type like service database. That's a CRD in Kubernetes terminology. Then they define how that maps to cloud resources.
Then, a service team can just write the manifest, the YAML that says, I need a service database and it needs to be this big and so on. Crossplane provisions and manages it for you. It's really cool because the service is defined at the Kubernetes level and now its infrastructure is defined at the Kubernetes level. Perhaps it's all packaged up into a Helm chart, so you install the Helm chart. You automatically get cloud resources in that environment. I think the technology is still maturing, but it's actually really interesting. Daniel Hertz company has one, Kratix, even better than Crossplane. Really interesting stuff, but it's deep technology.
The Observability Platform
Yet another platform I want to talk about is the observability platform. As you can imagine, a team that develops a service needs to understand how that service is behaving in production, and then they also need to understand what the users are doing, and how they're actually using that service. In the microservice architecture pattern language, there's actually six observability patterns. The gray ovals are the patterns. This includes the standard stuff like log aggregation, application metrics, and distributed tracing. It's like an observable service has to implement these patterns. As you can imagine, each of these observability patterns is some instrumentation which is comprised of application logic, plumbing, plus some dependencies, and that's emitting telemetry. Then there's infrastructure services like Prometheus, or something for storing logs and so on, that's gathering the telemetry, storing it, analyzing it, and presenting it to the teams. Yet again, on the one hand, the teams probably have to write some application-specific instrumentation, but everything else would be too much of a burden.
Setting things up, yet again, requires specific knowledge or in-depth expertise. Having an observability platform provide the infrastructure services like Prometheus, or ELK, or CloudWatch, or Datadog, so it could be a SaaS, or it could be on-prem, doesn't really matter, but it's stuff that needs to be provided, managed, and so on. Then, in the service template and chassis, there's elements that ensure that out of the box, the service is observable by default. You just build on top of the service chassis. You have an observable service. The team just needs to write the appropriate logging code, and collect the appropriate business-oriented metrics, and all of the low-level details are just insulated from them. Recurring theme. The team either ignores the low-level stuff completely, or focuses on the valuable parts, and everything else is provided via the chassis and these pre-managed infrastructure services.
The Build Platform
I now want to talk about the remaining two platforms, which are actually quite connected in a way, two sides of the same coin, the build platform, which is responsible for the deployment pipeline, and then the deployment platform that's responsible for the production environment. They're actually connected, because what comes out of the build platform actually has to be compatible with what the deployment platform is expected. Let's look at the build platform. Every service has a deployment pipeline that's going to compile the service, run the tests, package it up, perhaps as a deployable unit, like it could actually build a container, and a Helm chart, test that, and then either push it into the production environment or publish it to a registry where the production environment can pull it. Every service has one of these. It needs to be set up. It needs to be administered. That can be a significant burden for the team.
Then, also, the infrastructure that the deployment pipeline runs on also needs to be set up and administered as well. If we're using GitHub Actions, you've got to have a GitHub Actions workflow file for each service. Then the organization that contains the service's repository needs to be configured with GitHub Actions runners that actually execute the deployment pipeline, and also budgets need to be set and all of that administrative stuff. Not something you really want the teams to be doing, especially when you have a lot of services that are more or less built the same way, it's like, why reinvent the wheel? Good use case for having a build platform which provides the pipeline infrastructure, properly configured GitHub organizations. Then in the service template, there's actually a templated definition of the deployment pipeline, whether that's a GitHub Actions file or a CircleCI config.yml. Then, hopefully, the actual pipeline specification is built using these reusable package components like GitHub Custom Actions or CircleCI Orbs, to actually minimize the amount of copy and paste that's involved.
At least those two have a way of providing reusable deployment pipeline logic, and GitLab has something similar as well. Then the build platform can also provide some repository/registries for storing build artifacts, shared libraries, as well as publishing deployable units to a container registry for your container images and Helm charts. Because what comes out of the deployment pipeline obviously has to be stored somewhere. With this build platform, hopefully, most teams don't have to be concerned with all of this. It just happens for them. Maybe they need to tweak the deployment pipeline if they have specialized use cases, but hopefully not so much. I feel like this is pretty standard.
The Deployment Platform
Then that leads to the last part, which is the deployment platform. To actually run an application, you obviously need to have a production environment, and you might actually have some other environments, staging, dev, QA, though unclear exactly how many you really should have. Those are a centralized place where your service is run, and they actually have to provide a rich set of capabilities. They need to have a deployer mechanism that takes an updated service and rolls it out somehow. It's hard to talk about this in the abstract, but later on, I'll mention things like GitOps tooling like Flux CD or Argo. They also need to deploy your services in a fault-tolerant way. The big idea is like, here's my service, I need to run n instances of this. The infrastructure tries its hardest to make sure that's true. That could be an AWS autoscaling group, or it could be Kubernetes deployment, depending on what mechanism you're using.
Then there's also a networking capability as well. Requests that come in get routed and load balanced across your service instances. As you probably know, or maybe you're blissfully unaware, it's really complicated to set all of that up. Then the other part of this, every service needs some Infrastructure as Code configuration to define how it's deployed. It's either Kubernetes YAML, or maybe it's some Terraform Infrastructure as Code or something worse. There's a lot of stuff. It's really funny. I mostly live in the Spring world. When I step outside the Spring world and I go to the frontend, I'm horrified. I have been for many years. Then, over the past few years, I've been doing more of this DevOps-y stuff, and I'm equally as horrified. It is so complicated. I guess it's evolving. As a Spring developer, I don't want to have to deal with this. Actually, I find it cool and interesting, but there's another part of me that just wants to run away screaming.
You need a deployment platform to shield the developers from the complexity of the infrastructure. That will provide the various environments. Hopefully, in your service template, it'll provide IaC code for deploying the service, some kind of template. Then hopefully that is composed out of some reusable Infrastructure as Code components. Though the modularization technologies there leave a lot to be desired, specifically in the Kubernetes YAML world. Here's one example that I've worked with, which is where there's a GitOps tooling, here I've shown Flux, but there's also Argo CD. The state of the cluster is defined by one or more Git repositories. When those change, Flux will actually apply those changes to the cluster. It can also monitor the container registry. When a new Helm chart is published, it will edit the manifest and then deploy that new version. It's a pretty slick setup, but it is quite complicated. That's the deployment platform, and ideally it shields the developers from a lot of the complexity.
Microservices Platforms: How?
That's the six platforms. I just want to wrap up with a few comments about this platform development thing. While I was putting this talk together, I saw this article by The New Stack, which just pointed out that the success rate for platform engineering is not that great. Mostly fails. Based on what I've seen looking at organizations, is it's not surprising. I think one issue that's going on is for a lot of organizations, platforms are a way of ignoring the hard problems of people and solving actual customer problems. They do that by diving into what I call the infinity pool of technology. Because you can just get lost in the Kubernetes world and feel like you're doing useful things like creating custom Kubernetes operators. If you're not actually solving real problems, you're not going to succeed. A few thoughts. I've observed this antipattern forever. You really have to focus on the services, get those deployed in production, and worry about the technology later.
Specifically, what that means in terms of platforms, build some services, put them in production. Figure out what problems the teams are struggling with, then build the platforms, migrate the services to those platforms, and just iterate around that. Rather than go do a lot of platform development up front. Then team topologies has this great concept of a thinnest viable platform, or you could say it's like a minimum viable platform, where you build just enough platform to enable the teams to be successful. No more, no less. Maybe that involves Kubernetes operators, but maybe it doesn't. Then, teams need to adopt a customer-focused mindset, where customers in this case are the teams that are developing the services. The platform teams exist to help those teams, not actually dictate to them. They need to learn what their problems are, and provide platforms that help. Then they also need to actually help those teams use the platforms through facilitation. Build just enough platform to help, and do it in a customer-focused way.
Summary
In summary, microservices do require platforms to avoid the teams having to reinvent a lot of complex wheels. I identified six different platforms. There's probably more. Then I think it's really important to take a software engineering approach to these platforms, which are actually easier said than done with some aspects, and develop reusable components, rather than just copy-pasting everywhere, which creates a maintenance nightmare. Obviously, if you're a developer, you go, that's no big deal. Programming languages have libraries, frameworks, build tools like Maven, Gradle have plugins, even CI/CD platforms like GitHub Actions and CircleCI have reusable components. Then it gets a bit iffy when you're in the Kubernetes YAML world. There's Helm library charts, which are just weird. They work, but they're weird. Then there is a language called q, but that seems slightly weird to me as an application developer. I think there's work to be done in that area. You want to have these modular, reusable components everywhere. When you do have copy-paste, you then need tooling to actually automatically roll out updates to all of your services, generate pull requests for each one. Then, lastly, if you're building microservices, focus on the services, not on the technology. Build just minimal platforms that help.
See more presentations with transcripts
