BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Platform Engineering 101: What You Need to Know about This Hot New Trend

Platform Engineering 101: What You Need to Know about This Hot New Trend

This item in japanese

Key Takeaways

  • Successful platform teams have a mission statement that fits the overall goals of the organization, proactively defines the role of the team within the organization, and inspires your engineers. 
  • By conducting user research, soliciting user feedback, and getting internal buy-in from stakeholders, the platform team can gain a holistic understanding of developers’ pain points and shared challenges across the organization
  • Successful platform teams don’t reinvent the wheel. 
  • Internal developer portals provide a UI with service catalog capabilities that developers can use to access the underlying Internal Developer Platform. In contrast, Internal Developer Platforms remove toil and repetitive work and drive standardization by design.

With a rapidly growing community and tooling landscape, platform engineering is clearly here to stay. But, as with any (relatively) new trend, there still remain a lot of unanswered questions. Humanitec recently published the first State of Platform Engineering Report - here are the key things learnings from the report along with important trends in platform engineering.

Platform engineering is one of the biggest trends in software engineering. With PlatformCon 2022 - the first platform engineering conference ever - attracting over 6,000 attendees, and local meetup groups having memberships in the thousands, it’s clear that this trend is here to stay. 

As with any trend in software engineering, it’s only natural that there are a lot of unanswered questions that both people within and outside the community are curious about. The first State of Platform Engineering Report attempts to answer these ongoing questions, give information on the current state of platform engineering, and compile some key resources from different corners of the community. 

So, without any further ado, here are the key things you need to know about platform engineering. 

Why is platform engineering a significant trend?

In the last decade, complex microservice architectures, technologies like Kubernetes, and approaches like Infrastructure as Code (IaC) have become an industry standard. Even simple tasks now require developers to have an end-to-end understanding of their toolchain, vastly increasing their cognitive load and creating organizational inefficiencies like shadow operations.

Cognitive load is “the total amount of mental effort being used in the working memory.” Psychologist John Sweller started formulating cognitive load theory in the late 1980s. The theory asserts that “learning is hampered when working memory capacity is exceeded by a learning task.” 

In 2019, Manuel Pais and Matthew Skelton identified cognitive load as a major byproduct of an issue with current DevOps setups. This finding helped drive the creation of their Team Topologies, where a dedicated product team builds abstraction layers or platforms to shield developers from the complexity of underlying technologies and reduce cognitive load. 

Team Topologies was designed to combat specific, recurring antipatterns arising in some DevOps setups. In one anti-pattern we call “shadow ops”, which was confirmed by Humanitec’s 2021 DevOps Benchmarking Study, an engineering organization decides they no longer need dedicated operations professionals (or at least less of them) and shift those tasks to their developers. Oftentimes, this leads to less experienced developers leaning on senior backend engineers for assistance. This misallocates some of the organization’s most talented and important resources and decreases the productivity of the engineering team as a result.

Inspired by Daniel Bryant at PlatformCon 2022

As a result, some organizations were forced to find a better approach. They created platform engineering teams to build Internal Developer Platforms, or combinations of technologies and tools combined to lower cognitive load on developers without abstracting away the context of underlying technologies. Netflix used their platform to solve developers’ challenges to manage multiple services and software, knowing which tools exist, and switching contexts between tools. Nordstrom built a platform to become a “Northstar of self-service.” Their platform helped remove bottlenecks in their provisioning process and eliminate the need for people to log into production. These changes enabled the organization to move faster. Zalando leveraged their platform to unify the developer experience, promote compliance by default, and improve how the company operated over time.

These are just a few examples of successful platform initiatives. Studies like Puppet’s 2020 and 2021 State of DevOps Reports and Humanitec’s 2021 DevOps Benchmarking Report have shown a higher degree of DevOps evolution and performance on DORA metrics for teams using IDPs. 

So what exactly is platform engineering?

Agreeing on a common definition for platform engineering is difficult, but it’s an important task. I recently defined platform engineering as:

The discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an “Internal Developer Platform” covering the operational necessities of the entire lifecycle of an application.

Internal Developer Platforms are built by dedicated platform teams. Team Topologies discusses what this team is and isn’t. The platform team is not an SRE-stand-in or a shadow operations team that handles developer tickets. It should not focus on individual challenges or production reliability. Instead, it should be driven by a product mindset that treats developers as the platform’s customers. 

Platform engineering principles and best practices

Before platform teams can start building their product, they need to define a clear mission statement to guide the process. This mission statement should fit the overall goals of the organization and proactively define the role of the platform team within the organization. It should also inspire your engineers. Hashicorp’s Director of Platform Engineering Infrastructure Michael Galloway summarizes this well: “it should be emotional and inspiring. … It should be simple but meaningful.”

You can start by defining your goals. This could encompass things like enabling the required degree of developer self-service without adding cognitive load or achieving the desired reduction of tickets that go to ops without forcing developers to learn infrastructure-centric technologies end-to-end. After this, you’ll probably wind up with something like: “Our mission is to standardize workflows to improve the developer experience, speed up innovation cycles, and shorten time to market for the engineering organization.” It’s descriptive but not inspiring. Refine your mission statement to strike a good balance. For example: “Our mission is to build a platform developers love because it lets them innovate faster.”

The best way to build a platform is by following a product approach. By conducting user research, soliciting user feedback, and getting internal buy-in from stakeholders, the platform team can gain a holistic understanding of developers’ pain points and shared challenges across the organization. It can determine what features developers need and build a golden path that encompasses those solutions. But the platform journey doesn’t stop there. Successful platform teams keep open lines of communication with developers and measure engineering KPIs to make sure developers are adopting the platform and that the platform is making developers’ lives easier.

Lambros Charissis shared how his platform engineering team at Wise determined a set of actionable KPIs to identify challenges and measure performance. They used KPI trees to nail down their KPIs: productivity, lead time, deployment frequency, developer happiness, stability, efficiency, and risk.

You should also monitor platform adoption KPIs like market share (do developers actually use your platform or do they prefer to access their tech and tools directly?), hours saved by using the platform, and improvements in relationships between teams (like developers and operations, for example). Run regular surveys to figure out if your platform has improved the developer experience.

Platform teams and the golden paths they provide are the glue that holds these complex setups together. However, because platform teams don’t typically ship customer-facing features, many organizations incorrectly see them as cost centers. Platform teams should strive for internal buy-in from relevant stakeholder groups to ensure the longevity of their internal platform project.

Finally, and perhaps most importantly, successful platform teams don’t reinvent the wheel. Occasionally, the best solution will have to be built from scratch. However, many businesses are dedicating their resources to building specific and effective solutions. The tooling landscape is growing to address a wide range of problems. Platform teams can save time and create more value by tailoring off-the-shelf solutions whenever possible.

Platform tooling landscape

Platform engineering is about architecting an Internal Developer Platform. Kaspar von Grünberg explains that an Internal Developer Platform is “the sum of all tech and tools that a platform engineering team binds together to pave a golden path or paths.” But what tech and tools are out there? And how do you decide what works best for your organization?

There are many ways to build a platform and there is no one-size-fits-all solution.

The State of Platform Engineering report attempted to break down the platform tooling landscape to make more sense of all the options.

There are many ways to build a platform and there is no one-size-fits-all solution, despite what some vendors may promise. Some vendors provide a PaaS-like experience (like Heroku) that might work for small startups, especially those that want to push out a POC or MVP quickly. 

But for larger enterprise or also midsize engineering organizations that still need to support their brownfield setup and can’t start greenfield, in most cases, such a PaaS doesn’t work. 

However, platform engineering experts generally recommend following a platform as a product approach instead. This means you don’t always need to start from scratch. There are plenty of platform tooling offerings out there, many of them open source and some proprietary. Where possible, consider buying and customizing pre-built solutions to save your organization time and money. This leaves time and resources for your platform team to dedicate to features your organization needs custom-made.

The State of Platform Engineering Report attempted to break down the platform tooling landscape to make more sense of all the options.

In the report “A Software Engineering Leader’s Guide to Improving Developer Experience,” Gartner outlines three key ideas guiding platform engineering:

  1. Improve developer experience by building internal developer platforms to reduce cognitive load, developer toil, and repetitive manual work.
  2. Platforms don’t enforce a specific toolset or approach – it is about making it easy for developers to build and deliver software while not abstracting away useful and differentiated capabilities of the underlying core services
  3. Platform engineering teams treat platforms as a product (used by developers) and design the platform to be consumed in a self-service manner.

It’s also worth clarifying the difference between Internal Developer Platforms and internal developer portals. Gartner’s report also addresses this subject. The authors write: 

“internal developer portals serve as the interface through which developers can discover and access internal developer platform capabilities.” 

Internal developer portals provide a UI with service catalog capabilities that developers can use to access the underlying Internal Developer Platform. In contrast, Internal Developer Platforms remove toil and repetitive work. 

Building a developer portal won’t solve the underlying configuration and workflow issues that are negatively impacting your developer experience. That’s why a combination of an Internal Developer Platform and a developer portal is ideal.

Community and careers

The Platform Engineering community started in 2021 with a handful of meetup groups in Austin and Berlin. Today, it boasts over 10,000 active platform engineers across 19 meetup groups around the globe. This community momentum is a sign for organizations to take platform engineering more seriously.

Platform engineering is also a promising career path. Many companies are realizing that building Internal Developer Platforms adds value to their bottom line. It follows that people who build Internal Developer Platforms could make more on average than their DevOps counterparts. 

Despite this shift, job titles like Platform Engineer or Head of Platform aren’t yet ubiquitous. The State of Platform Engineering report found that most respondents who were building their organization’s platform had titles like Senior Software Engineer, IT Architect, Principal Engineer, or Senior DevOps Engineer. So while the industry is starting to embrace platform engineering, it still struggles to define platform roles appropriately.

Folks considering a platform engineering career often ask the community which technologies they need to master. In our survey, more than half of the respondents reported focusing most on Docker, Kubernetes, IaC, and CI/CD. Database, storage, and networking technologies also made the list, but for less than a quarter of respondents.

When it comes to flexible work arrangements, platform engineering is ahead of the curve. Over 80% of survey respondents from the US are fully remote. European respondents have a more balanced split between fully remote and hybrid setups. Overall, 100% in-office is 100% a thing of the past.

What’s next?

Platform engineering has come a long way over the past few years, and it isn’t going to slow down anytime soon. With the tooling landscape and community growing day by day, there’s no shortage of resources that can help you get started with platform engineering or further your career.

About the Author

Rate this Article

Adoption
Style

BT