Transcript
Borst: My name is Jelmer Borst, working for Picnic Technologies as a PM for our platform products. I would love to talk about how to apply a product mindset to your platform product tomorrow. Some of you might know Ben Hunt-Davis. He's the coach of the UK rowing team. He primarily focused on, will it make the boat go faster? As a result of him going into the Olympics, they had a big challenge in order to figure out how to really become fast. His main thought was, if we apply all the different elements, the idea of making the boat go faster, so for every choice you had to make, the question was, should we do this to make the boat go faster? Very similarly, that relates to how platform products work. Namely, what can we do to make the boat go faster? What are the elements we can do for all the other product teams out there to accelerate them? Very often, you start like this. Especially coming from startups, in my case, six years ago, when Picnic was still a small company with only a handful of engineers, and now being a size of 35 product teams, is that you start with building all these awesome, new features. This goes super well. You're scaling up. You're interviewing your customers. You're figuring out what to build, and you keep building more features. That goes actually well for quite some time.
As many teams know, very soon you start looking into things like technical debt. Technical debt is often taught in the elements of architecture. What is holding us back? Because we implemented this feature, but now we have more requirements coming up, we have this new idea that we can skill the better. How we implemented the new architecture that we have, this has allowed me to skill. Or maybe we need to do some refactoring in order to actually have a better experience as a developer. Very similarly, that also starts applying to maybe your CI system or your CD system. How you're going to do a release is maybe not so important if you're just a handful of engineers, so let's say a team of five. It starts becoming very important if you're a couple of hundred or a couple of thousand engineers who want to release many times on a given day. Especially you're scaling your total tech team. In order to solve this, you think, let's start a platform team. Let's start to have this team that focuses on solving a lot of these common problems. That isn't easy. You may have super smart engineers working on this platform team who's trying to build the best possible stack for the others to build upon. How that's being used is not always easy, especially if you're in multiple countries, or you may be having a remote team, or maybe there's just a lot of engineers, they have a lot of product teams, you might not actually know how your product is actually being used. Therefore, maybe it's gut based, or maybe it's what some other developer told you, where you're deciding, should I build feature A or feature B, or should I improve my system in a certain way?
I love this small comic, where they're going through this process where, we need to make 500 holes in the wall, so I've built this automatic drill. It uses elegant precision gears to continually adjust its torque and speed as needed. You've designed this amazing system that works so well for what you think it requires. You might even think we've potentially over-engineered this. Where the other guy says, great, it's the perfect weight, we'll load 500 of them into a canon and we just shoot them at the wall. They completely ignore your completely over-designed automatic drill. He just wants to accomplish by demolishing that wall, or basically getting in that. We see this on a daily basis where, a lot of other engineers don't care about how maybe you build, or test, or how you release, or how we could, in the ideal scenario, get a couple of megabytes down from your Docker image or something. Where they just want to ship features. That means that we need to think about how can we enable them to do this best, and not by building over-engineered solutions that in the end doesn't actually help them. In this example, you could just have given them a couple of stones or bricks to solve the same problem, and have a ton of time available to actually do something else.
What Is Platform Engineering?
To maybe start off with, what is actually platform engineering? We just said, let's build this platform team. It seems to be coming up more with different companies doing this. We have a central team that is now going to solve all our problems. What is it actually? As a definition, platform engineering is the discipline of designing, building toolchains, workflows that enable self-service capabilities for software engineering organizations. Let's start breaking that down. That means we're designing something. We are building something for toolchains and workflows. We're primarily going to focus on the designing part. Figuring out, what should we build? How should we build this? How is this going to be used, so that you actually figure out the best solution? Then the second part is actually building those solutions. Most platform teams, or these central infrastructure teams are really actually super good at building. That's usually not really the problem, therefore, we would love to focus primarily on the former.
Then the second part is super interesting as well. We're talking about toolchain, so internal tools, often. Secondly, there's also external tools that you're using as an organization. Whereas maybe on the get-go, you're not thinking this is your job to actually manage or use them or configure them appropriately, because it's only being used by two out of these many teams. There's maybe not too much use of it. However, it can actually be quite crucial for the success or the productivity of all our engineers to also include external tools. Similar for workflows. Very often, we think about platform engineering being just on tooling, but actually, there's a huge focus also on workflow. How is one of my internal-external tool actually being adopted? Also, what is the process of the different product teams out there that are actually using my tools on a daily basis? Not only the tools that you provide, but also how do they run in general, their weekly, monthly, quarterly processes? Whereas in many cases, there's actually a lot of things you can do to accelerate teams by not actually doing something around tooling, but actually focusing on their processes and help them.
Product Mindset for Platform Teams
I would love to give some ideas of just product mindset for platform teams for everyone out there. On one hand for leadership, who's figuring out how can I actually get the most value out of these platform teams? Maybe you already have a platform team or an infrastructure team, but you feel you could do so much more, or you feel they're not going quick enough or not having the impact that you would expect? Or maybe you're thinking about actually starting a platform team. It is also for engineering managers who want to figure out, how can I empower my engineers on my platform or infrastructure teams to better decide and improve their product, and improve their way of working? You see more platform teams who have a product manager either part-time or full-time on their team, and especially for companies where this is newer, maybe scaling up, you're starting in this team, and maybe internally you're transitioning into this role. Suddenly, now that means actually that you have quite a different product at your hands. Maybe you're quite new, either as a PM, or maybe just new in that particular role, so how can you find your way in this overwhelming space? Then the last one, I think, which is super good to call out is engineers who are on a product team where maybe you have a PM, maybe you don't have a PM, where you want to improve your impact, you want to improve what to work on, both on a platform team, but also actually on any other team in your company. Where very often the PM on your team is maybe figuring out together with their stakeholders what to build, and maybe you feel that that is the [inaudible 00:11:37] you need to deliver on, whereas, actually, there's a lot of value in accelerating your own team as well. Thinking, what are the things you think about to understand where you could improve as a product team as being an engineer or maybe a technical lead of that team.
Very high level, I want to give you five different takeaways you can actually use. First of all is stop building. This seems very counterintuitive, and I don't mean, go on a strike, stop doing literally anything you're doing. Stop and think a moment and start talking to your users instead. Secondly, is aligning on the company's strategy. Understanding how is your team fitting into the direction of the company, such that you can have maximum impact as a team, within the scope of your company. Third is to finally explain also your team value to upper management, to other product teams, for them to understand what you're doing, and how they can also use and leverage you for their own successes. Fourth is looking into measuring what matters and ignoring everything that does not. Figuring out what to measure, how to even conceptually think about this. There's so many different articles and books floating around, and measurements and metrics, and it's so easy to get stuck into the gazillion number of ways of measuring. How to do this within the scope of your product. Lastly, and which we should never forget, is iterating and celebrating all sort of successes.
1. Do Not Build
To start off, I would like to call out to stop building. As an engineer or as a team, and even for PMs this is sometimes actually quite hard to do, is you have this awesome idea in your head. You're working on a day, you're hitting a certain issue, or maybe you had lunch with a friend, and you're discussing a tricky issue. As a result, you're very prone to actually start solving that. As engineers we love to solve and build solutions. You're prone to actually figuring out, how can I open up a pull request to make this change? It only maybe takes you an hour or two, and we're helping that one person on this other team. Where very often because we're so prone for action and trying to solve that particular problem, we're not really solving the large problem at hand. Or maybe we're only solving for one person we're not solving for basically all other product teams. As a first one is to start talking to people. You want to start talking to your CTO or your CPO to understand what they see as being main struggles. Please don't stop there. In some cases, we get this top-down, this is an issue, where should we start? We should solve this. Of course, that usually means we should absolutely look into it, but again, don't start building straightaway. Secondly, start talking to your users. Your engineers, technical leads, engineering managers, architects, other product managers, what is holding them back, what is causing them issues? Having these interviews. Do this structurally. Don't just have a chat, put some stuff in your brain, but really have a focus on building up the structured interviews where you're going out to your users. Separate them from junior engineers to senior engineers, to see what type of problems they are having. People who are very new at your company versus people who are actually very long at your company. They all have different problems that they would love for you to solve for.
Many teams are still in this ticketing mode, where another team is hitting an issue, they're trying to solve something. That just comes throughout the quarter, they're trying to deliver on their own roadmap. Now they want you to support whatever they are focusing on. They create this ticket, and in some cases, it is like an actual ticket for maybe an infra team to pick up and solve. In other cases, it's not really like a Jira ticket, but it's actually a feature request that comes through and suddenly has this very high pressure to solve. We all know these examples where they come in, we need you to solve this and support this in the next coming days, because we're blocking this project, and we have so much pressure to deliver, and we really need to do this. While you would obviously love to help them, maybe you're cutting corners to support a feature where this would actually be very unrealistic to be able to do actually in a couple of days. How can we start thinking about all of this more proactively? Apart from making your own work a lot more structured, it also means that it's not only about the most vocal engineers. If others are hitting roadblocks, they're reaching out to you. They are the most vocal engineers that are unafraid to step up. They are also actually the ones that may be either are a little bit more senior, which is often the case, or perhaps, let's say, some who are just very loud with sharing the ideas of how something should be done, or what is required. I'm sure it is very recognizable for a couple of people that you come across very often. Whereas, actually, you want to not solve for the most vocal engineers, or vocal people, or maybe your CEO, but you want to solve for everyone so that you can have the biggest impact.
It's very cliche, but start designing your personas, who are your users? Say you're on a team who's maintaining and improving the company CI system, and you have these different audiences within your company. On one hand, we're supporting Java engineers, QAs, and mobile engineers. In this case, so we have only, let's say, a backend in Java, we have only a mobile app that mobile engineers are working on, be it iOS, Android, or maybe something like React Native that is being supported. We have QAs to support ensuring quality that we're putting live. Your CI system is being used by effectively all three, either directly or indirectly. Next to defining who are you effectively building for, it can also be helpful, who are you effectively not building for? Especially if there are some users who are somewhat involved, maybe from time to time, they might use it, to make this explicit that you're not building for them. It can actually be very fruitful to also articulate to them, I know you're also using our system from time to time, but you're not our primary audience, and that is why we are solving these needs. Then you can have the discussion if they agree and if that makes any sense.
As a second step, don't use these imaginary personas. Start tying them to actual people. These are your most advocates that you want to look out for. If you think about the Java engineer in your company, who is that? Who is somebody who is already used to thinking about how we can improve? That person is very helpful, maybe in collaboration, maybe you're working on some pull requests with them before. Here we have, for example, Steve on the payments team, that you could work with a lot for improving CI for Java. Then we have Abby on the promotions team, and Liza on the webstore team. Making these actual people allows you to have that discussion with them on certain features, allows you to have this person that you can first start using with interview, but also has a sparring partner as you go. People have opinions, and that can be actually quite scary. As Kathy Korevec said, you're a chef cooking for other chefs. You are an engineer building for other engineers. That can be rather scary, because they will have opinions of what you've built. They could have theoretically maybe also built this themselves, so that means that very often they will voice their concerns or maybe voice their opinions that maybe is not good enough. Really try to embrace this. Really try to focus on getting their input as much as possible. If it's not good enough, and if they are very vocal about what you're putting out, that is actually super good.
2. Align on Your Company's Strategy
Secondly, is aligning on your company's strategy. Understanding what the success of the company depends upon. Do you maybe have large productivity issues? That means that engineering enablement becomes actually key for your company to succeed. Maybe there are hiring challenges. Maybe you can't offer the same compensation as the FAANG companies out there. Or you might have scaling issues as you go. There's exponential forecasted growth, so how can you from system side properly support this? Or even reliability issues that you have, primarily, maybe you are building the SaaS solution, where towards your customers, you want to make sure reliability is key. That might be actually a theme at the moment. This seems actually very easy in the sense of, obviously, we have these issues so therefore we should do work on them. Trying to think of, as your company to succeed, as you in a year's time or a couple of years' time what becomes really the elements that you as a platform team, or engineering enablement team, or infrastructure team can really do to enhance and empower that. Relating that to your team and explaining that can actually help on one hand getting the right resources in place to do that, but also actually means you start having a bigger impact. Also, for yourself, actually that recognition or that idea of success can actually help a lot in how you like doing your work, and how much you can do.
For Picnic, we are an online groceries company, our model is where you can order today and we deliver it tomorrow. This is not your quick e-commerce that you order now you get it in 15 to 30 minutes. This is really for your weekly groceries. We're super customer centered, to reduce the time that you're waiting in line. Really getting the boring elements out of somebody's life, because nobody enjoys sitting in traffic after work to stop by the grocery shop. Nobody loves spending their time because you're going to want to be with family, with kids doing other joyful things or hobbies that you want to do, where groceries for most people is not one of those. We have this huge sustainability potential by drastically reducing waste, as we can optimize for what customers want to buy, or what customers are buying instead of trying to predict what they might want to be buying. In order to do this, this is actually not very trivial, so we need to build a lot of internal systems to make this work. Namely, as a supermarket, our margins are razor thin, and in order to still be able to have this whole model to work, we need to basically build every essential piece of tech ourselves. We have our own customer app, which is very sensible, so users can download our app, log in, and order their groceries. However, the entire supply chain is also managed by us. We have our own ERP, our own users, deliveries, orders, we have our own management warehouse system, we have our own route optimization software, or delivery app, our forecasting. We have stellar customer service that we built a lot of it ourselves as well, to actually make it a lot more customer friendly, compared to the most traditional support. It's out there. The cherry on cake, we launched this automated warehouse about a year ago, with all software built internally.
What does this really mean? This is a nice story about my company, about Picnic. How does this relate to thinking about strategy? First of all, this means we have many internal products with high complexity and a large domain. This actually leads to understanding, what are the elements you should be doing as this platform team to be able to empower this? You have on one hand engineers who are very deeply focused on their own product, which means that actually linking it to, or making it easier for them to get really into the nitty-gritty to solve for their problems is key. It also means we have a ton of different systems out there, where you're working with other services that you're not so familiar with. That actually also requires quite a bit of domain, not sure how are you navigating that space. What if you have an incident, or what if you want to integrate with this other system? Understanding how your landscape from a tech point of view works is actually quite key for your product, to understand where you can actually have the most impact.
Somebody made this analogy of customer facing product managers versus platform product managers, where, on one hand, the customer facing PM has this focus for this one goal. You have this key metric that you're trying to solve, or this key need of a user that you're really trying to optimize. As a platform PM, sure you're building for your users, but actually you're building the right tools and workflows to achieve the customer facing PM's goal, because it's not about your goal. Because if your platform team is doing amazing, but the company is still going down, then you're not doing a good job. In the end, you're trying to solve for the whole company. It means this becomes more a SimCity exercise where you are trying to figure out what they want to achieve such that you can actually support them the best that you can. To me, it's similar to having the NorthStar metric where the customer facing PM is trying to build towards and working as best as they can to reach their destination. The analogy is, if you're hiking in the mountain, they're trying to hike as fast as they can in order to reach their destination. As a platform, you have this different view, where you're standing behind them and see them walking there. You need to think of, what can I do to help them reach their destination faster? Now you're thinking about, I maybe need to build or invent a compass such that they will reach their destination faster, or maybe having some form of maps that we can actually start really using. Obviously, this sounds very straightforward and easy. Having this separate view to understand what are they doing, where do they want to go, so what can I do to help them?
3. Define and Explain Your Team's Value
Thirdly is to define and explain your team's value. We were talking earlier on about talking with your users, and internal partners are effectively a little bit more than just your users. You have a customer base of people who are using your services, as a platform team, you're really trying to look for these internal partners that you can actually start working with. It changes from this question-answer type of mode, or doing this interview, really to do collaboration. Remember these people that you have defined as your personas, these are the perfect people you can actually use as your internal partners. That is on one hand for them to collaborate with you to solve something. At the same time, this helps a lot for them to understand, what is the value that you're adding to the company? When should they reach out, when they could actually use you? Instead of trying to solve something themselves, they could now start reaching out to you to solve their need.
Secondly, is also trying to educate your leadership, being this voice of your users towards also leadership. Why is it complex for them to build their solutions? What makes the software delivery hard? What are the challenges others are facing, day-to-day? What are the bottlenecks that they have, that you can solve for and that you can help them out with? What you see very often is that many people or many engineers in your company, are having these issues that somehow do not bubble, or do not nicely get prioritized, or are not greatly voiced or articulated, because it's so distributed because you have different pockets with different people having different problems. At the same time, they might not always actually voice this in maybe a public channel or otherwise. Also, it is very often this death by a thousand cuts, so all these small little issues that are, on its own, not too much of something, but it's really this cry for help. On a whole, all these small issues actually lead to quite some impact to you as an organization.
Then, also, manage expectations. Especially if you're starting out a platform team, you might not have solved all the problems tomorrow, even though you think, now we have this couple of people focused on it, then easily we should be 10 times faster tomorrow, or let's say in a week or a month's time. It's quite a long-term investment. It has an amazing high return on investment, but you are doing something for the long term. The fact that you're building something now will help, let's say, building that new feature tomorrow, next week, or next month. However, even though it has this longer-term investment, don't use it for yourself as an excuse actually to go slow, which is the tricky part. On one hand, don't try to oversell that you have solved all the problems tomorrow, but at the same time, also, for yourself still set the bar high to improve.
4. Measure What Matters
One of the key elements of being a product team is to measure what you're doing, and for that reason to iterate and learn and understand if you're adding value, and articulate for issues that you have. What you see in many companies is that product teams start either measuring literally everything that you could potentially be measuring, or are maybe measuring very few. You really want to focus on a couple of key metrics, and really start ignoring everything else. It's so easy for people to come around and ask you for this one single feature, or one single request, or the small thing that you might need. Like, yes, we can probably support it. Even though it might actually not help getting you towards your goal. Stop doing favors. If it doesn't make the boat go faster, just don't do it.
In many product teams where people are talking about platform teams, very often come out and say let's just use DORA metric. I know DORA is already a little bit outdated in this regard, but you still see this so often, where the DORA metrics solve for. For those who have maybe lived under a rock and have never seen this, it has these four key metrics. Over time, they also added a fifth and a sixth metric. Primarily, it's about deployment frequency. It's lead time for change. How quickly is your commit in prod? The time to restoring your service, often also measured as mean time to restore. If you have an incident, how quickly do you actually resolve this issue? Your change failure rate, so how often do you actually have issues in production? I've seen these four metrics which are related to a company's success from an engineering point of view. Many would argue, let's just start there, throw it on there and start optimizing. The question is, really, what are you optimizing for?
Don't start with the metrics. It's so easy to think about, we should track uptime, or we should actually start increasing our deployment frequency, because that's what DORA says. Or maybe you've read an article from another company who are doing this, and they're doing it well. You may need to, but it really depends on what you're trying to solve as a company. If deployments are really holding you back, you might actually want to solve for deployment frequency, but maybe that's actually not your issue at the moment, and you should actually be solving maybe for reliability, or something else. First, really start with the strategy from your company, but also strategy from your team. What problem are you really trying to solve here? Then, secondly, what do we aim to achieve with it?
There are four areas that you can think of, that you want to measure for. On one hand, measure to plan. That is really your product impact, your mid to long-term metrics, where maybe this could be issue lead time, or maybe the number of projects that are on time. Really, not something that you will change, influence directly tomorrow, but something that's a little bit over a longer period of time. Let's say maybe the six-month timeframe. Secondly, is measuring for the board. Your CEO or the finance team, or maybe your CTO or CPO that are interested in the impact that you're making. That can be from infrastructure costs, or to sort of a company might change its shipped to the metrics that for them give an understanding of what you are delivering in terms of value directly to the company. Obviously, this relates to the part we already talked about, in allocating them, where, let's say, talking about cost is a very easy one that you can start with, because your finance team wants to maybe reduce cost. As you go, you might want to actually include some other metrics, as you allocate them.
The third is to measure to optimize. This is really for your team, internally, where you want to verify the experiments, like the features that you're building, and you're launching, where you want to see actually some meaningful improvements. This could be incident rate. This could be maybe the number of canary releases that you're doing. That really depends on the challenges that you have and that you're solving for. Then, the last category is to measure to operate. This is very in this one-to-two-week cycle where you want to think about this as observability for your product. Maybe it's uptime error rate, maybe it's other things. Really something that is fluctuating where you would really see as the solutions that you're building, or maybe problems that come up that you maybe need to respond for. These on one hand can be metrics for your own team, but very often these are measures for the whole organization. This is why on one hand this is not easy. Also, this is actually quite crucial to get this right.
Having these four categories in mind, how can we actually define the metrics that we need to go for? A very helpful way of thinking about this is to start with outcome, so really this NorthStar metric of your company. Secondly, output, which are lagging metrics, which are directly related to solving that NorthStar, but still lag quite a bit. This is not a metric you will directly influence tomorrow. However, these are driven by input metrics, these are leading metrics that you can actually influence with projects that you're doing. I put here an example for an e-commerce company, like Picnic is, that you have your number of deliveries which is our NorthStar where we are trying to optimize it and where we are, and as Picnic is growing, where it is primarily driven by first order conversion and retention. These are very lagging metrics. This is a result of so many things that is happening that result in this first order conversion, or this retention of our customers. Retention might be actually driven very much by repurchases, maybe recipes, and some others.
If we're looking at the leading space, or input metrics is potentially, or article ranking, or our search, which is actually driving the repurchases that customers are doing. These would be our leading metrics. These are the ones we can actually directly influence. Very similarly, we can also do this for platform teams. The number of deliveries. We still are not suddenly solving for a different outcome, it's still the same outcome that we were solving for, because we are looking for the success of the company. We're still focusing on number of deliveries, but our output metrics start changing. Our lagging metrics could be maybe quality of service, and maybe your iteration speed, which is very high level. The faster we can actually iterate, the faster we can actually grow. The faster the other product teams can actually put improvements down that will actually increase the number of deliveries. That is mainly driven by how quickly we deploy, your issue lead time, and all kinds of different factors that will contribute to it. Looking at those, which ones are important? Perhaps deployment frequency is actually one that is actually driving your iteration speed and is actually rather low, instead of trying to push for improving and start understanding the underlying drivers. This is driven by batch size, this is maybe driven by the number of canary releases you have, that is only done in a very small portion of your teams. These are metrics you can actually influence directly with different projects you're doing, either on the adoption side or actually on the tooling side.
Now that you figured out a way how you can define the metrics that you should be using, the question is, how should you actually use them? Build a habit to review metrics periodically, so to plan every quarter. For the board, it's roughly every quarter. Optimize every two to four weeks. Operate every week. You have this different cadence for the different type of metrics out there. Build hypotheses why these metrics change. Verify your hypotheses with your internal partners to understand, I see this issue happening, does it actually make any sense? I think it's due to this. What do you think? What are you seeing over on your databases? Now that also, because very often these metrics are quite aggregated, start slicing and dicing, make segments of different users, different tenure throughout your company, to understand what's happening. Lastly, align these metrics with reality. If in reality, there are issues, but you don't see this in your metric, something is wrong. If your metrics are showing issues, but in reality everything seems fine, it's very likely also something is wrong.
5. Iterate and Celebrate Successes
Now's the time to really put everything together. You are measuring the key metrics for your product based on iterating there. You are learning what is happening. You are talking to your users. You start understanding fluctuations, what is driving your overall goals. Based on that you can start building improvements. That is actually not necessarily easy. This iteration cycle is required to make sure what you're building actually matches with your user expectations. Don't throw your product over the fence, where now I've built this improvement in the CI system, please adopt. Tools are amazing, but without adoption they're nothing. We too often actually try to use developer tools as spaghetti. We throw them at the wall, and we try to see what sticks. In our track, there's also this great talk by Olga, who's focusing actually on adoption. You might want to check that out if you're more interested in this area.
Lastly, share success stories. Have regular updates, for example, Slack, or your internal communication system. Maintain a changelog of what you're building, for people to go back and understand what you've done. Join all-hands to share the wins, but also encourage your users to share their stories, who are using your product. Lastly, be transparent also on your challenges. Nothing is going to be easy. Don't try to show that everything is going super well, if it's not. Be transparent on what the challenges are and what you're trying to solve as a team but also as a company. Where that helps for others to also understand where you're going, where is your focus as a platform team, to making sure that that is also aligned. Those were my recommendations of applying product mindset for your platform team. Don't start building straightaway. Align on your company's strategy. Define and explain your team's value. Measure what matters. Iterate and celebrate your successes.
See more presentations with transcripts
