In this podcast Shane Hastie, Lead Editor for Culture & Methods spoke to Pia Nilsson, Lead Platform Developer, Experience Lead at Spotify about what it takes to create a great developer experience.
Key Takeaways
- Developer effectiveness is about enabling flow
- Rather than mandating tools, build products which make developers roles easier and they will want to adopt them
- Rumour driven development is the state where finding out about available tools, APIs or other elements depends on knowing who to ask rather than being able to find the information you need to do your job effectively
- Identifying a set of metrics which provide useful feedback and valuable insights is difficult
Subscribe on:
Transcript
Shane Hastie: Hey folks. Before we get into today's podcast, I wanted to share that InfoQ's international software development conference, QCon will be back in San Francisco from October two to six. QCon will share real world technical talks from innovative senior software development practitioners on applying emerging patterns and practices to address current challenges. Learn more at qconsf.com. We hope to see you there.
Good day, folks. This is Shane Hastie for the InfoQ Engineering Culture podcast. Today I'm sitting down across many, many miles with Pia Nilsson. Pia is the lead platform developer, experience lead at Spotify. Did I get that right Pia?
Pia Nilsson: That's right.
Shane Hastie: Welcome. Thanks very much for taking the time to talk to us today. The place I like to start with my guests is who's Pia?
Introductions [00:58]
Pia Nilsson: Thanks for having me. Delighted to be on the podcast. So who am I? I've been an engineer for 14 years and then I went into leadership, so about seven years of that. I think my passion is platform engineering or has become over the years platform engineering because I sort of during my engineering years got tired of re-implementing stuff. So the duplication became very evident to me and I wanted to help. So I gravitated towards working in the DevOps space. So with CICDs, I joined Spotify to lead the CICD team, which is a passion of mine as well. So I think that's who I am.
Shane Hastie: What is developer effectiveness?
Developer effectiveness is about flow [01:42]
Pia Nilsson: Oh, that's a fantastic question. I want to say flow, because if you go back to yourself, when do you feel effective? I believe it's when you are informed about the context around you and enabled to keep your flow. So we do measure this at Spotify in our engineering satisfaction survey every quarter, and this is a real metric that we sort of try to help us inform decisions on, are these tools that we're providing to our engineers actually helping them remain in flow or are they being interrupted?
Shane Hastie: Are the tools helping with flow? To delve into the cause-effect relationship here? How do you know? Is it the tool or is it something else?
Pia Nilsson: Well, exactly. It is not only the tools, because flow is something that is created within sort of a team that you're in, and the context you're in, and the tools you're working with. It's all of that at play together. So what we of course have discovered is psychological safety of course matters a whole lot for a team to be effective and also just in general interruptions and being empowered matters a lot. So we have a very strong culture of autonomy at Spotify and I believe that is something that I have started to really connect with. People who feel empowered usually are more often in flow than people who aren't feeling empowered and it's a mix of so many different things.
Shane Hastie: One of the things I've noticed working with organizations, almost to the point of being a bit cynical, psychological safety has become a buzzword. Of course we have a psychologically safe environment, you are empowered, but you made a strong statement there. People that feel empowered versus what's being said. And I've often seen that, of course my people are empowered, when actually they don't really feel that way. So how do you make it real?
Building products developers want to adopt [03:49]
Pia Nilsson: Yes, that's a great point. So I totally agree with, it's a challenge for us as an industry, I believe, to really nail. Well, how do we get there? Everyone believes in psychological safety, but how do we actually get there in an authentic way? I think we sort of stumbled upon it a bit at Spotify. So I have seen one way of getting there in an authentic way. As a platform organization at Spotify, we didn't have the mandate to say to our 4,000 engineers, these are the tools you're going to use. These are the standards you're going to follow. That mandate did not exist actually to a very large extent. You have to have some mandate when it comes to some sort of compliance and security of course, but then there is a lot more than that and there was no mandate. It was pure autonomy basically, which led to a technical sprawl I had never seen before.
So the lack of mandate for a platform organization that care about rolling out best practices led us to building products that people wanted to adopt because that was the only way we could affect change in the engineering population. Well, if you need to build products that people want to adopt, you automatically become customer oriented and you have empathy for your customers. And also we kind of, of course, identified with the customer a whole lot because we were using our products ourselves, we were applying them to ourselves, which also dogfooding is also I think a really good way of getting there in an authentic way.
Shane Hastie: Oh, dogfooding. I so much prefer drinking our own champagne. I don't want to eat any dog food personally. So thinking of that, drinking our own champagne, using the tools ourselves and building that empathy. When your customers are technologists, that's a very, very different... Or is it a different mindset to thinking about the consumer customer?
Pia Nilsson: I would really think so, Yes. I think the people we recruit, the highly skilled engineers, the product managers, engineering managers and so forth, they need to co-create our solutions because they come in and join our organization and they kind of know how to be effective already. So I think it's very important to not think that we sort of have the solution and then we're going to help people use it, because I think it's way more of a mutual relationship when you're building tools for an engineering population, because effectiveness is a little bit elusive, so it needs to be co-created. Maybe that sounds a little too fluffy, but I think this simple need of actually enjoying using a tool, enjoying work matters a whole lot when it comes to actually being in flow.
Shane Hastie: Enjoying work, but we want people to be busy. We want to use their time efficiently. Or do we?
Pia Nilsson: That's a very common, I feel, misunderstanding. I held that belief myself in the early days of my career. I was very, very busy. And then I see when we have hack weeks at Spotify, I'm not just speaking about the annual hack week, we have often hack weeks on smaller levels as well. There are just so many great things coming out of hack week. It's like the hive mind all of a sudden produce these great features and ideas. I've heard people speak about over a coffee, but nobody had time to take a look at it if it actually is some substance in this idea. So I'm sort of realizing that effectiveness is when you actually let someone who is passionate about what they're doing actually think freely more often. We can take leaps instead of steps. That's how I feel. I kind of want to have hack weeks, I don't know, at least once a quarter. It's incredibly effective I feel.
Shane Hastie: I like that. Taking leaps instead of steps. One of the things that you've been very involved in is Backstage, which Spotify has released into the wilds and makes freely available. Tell us a little bit about that.
Introducing Backstage [06:08]
Pia Nilsson: Yes, Backstage was launched in 2018 at Spotify, built upon an already existing system that was only targeting backend services at the time. So out of that grew Backstage and we created it because there was such a clear need from our engineers that they couldn't find the things in our infrastructure anymore. This technical sprawl had become just quite overwhelming. We had an onboarding time that we measure over 60 days, which our target date was 20 and that onboarding metric is measured like how many days does it take an engineer to merge their 10th pull request. At the time, it was a widely spread metric to use, so we had borrowed it from some of our peers. So people couldn't find things and also people were interrupted very often. They were saying, because we guessed that was because other people couldn't find things and tapping these folks on the shoulder.
We called it, we had rumour driven development because you actually had to speak to someone in order to understand how to integrate with a certain API, which of these APIs actually finding the docs, et cetera, finding the updated docs if there were any, et cetera. So we really relied heavily on personal connections and you can also imagine that also is not very inclusive. So unintentionally that kind of culture with a lack of documentation, lack of clear interfaces, it wasn't easy to onboard, it wasn't easy to feel included and empowered in this environment. So out of that grew Backstage sort of. We wanted to democratize knowledge about the infrastructure, so you should be able to find, create, and manage your components, your team's components without having to lean on anyone really for these very simple sort of obvious tasks that everyone wants to do on the first day at work. So that's how Backstage was created.
Shane Hastie: And what's happened since you've released it into the wild, into the community?
Pia Nilsson: Yes, it's been fantastic. We decided to open source at 2020 because we were relying on it so heavily internally and we needed to protect it from another industry standard coming along, which we knew we would then have to migrate to, as Backstage was an internally built developer portal. So we open sourced and donated it to the CNCF to protect ourselves from a migration. And then the interest in Backstage just skyrocketed. It's fantastic. Today we have, I don't know, 1,300 customers that are using Backstage in production. I mean, since 2020. So they are now pretty mature Backstage users, many of these companies. And we estimate it's like 1.4 million developers are using Backstage right now. So the interest is just massive and we're incredibly happy about this. So we are spending a lot of time supporting the open source product.
Shane Hastie: One of the things we were chatting about earlier is the silver bullet thinking that is still prevalent in the industry that we'll install something, so we'll install Backstage, magically a miracle will occur and our developer experience will be magnificent. Of course, that's not the reality. What are the steps? How do we get from, oh, we want this product to, we've actually got improved developer experience?
Improving developer experience is a journey over time, there is no silver bullet [11:29]
Pia Nilsson: Yes, that's a lovely question. I think the more we as an industry think about what are the concrete challenges in my organization that my engineering population are experiencing, the more successful we would be with any developer tool really. So when it comes to Backstage, even though I'm really convinced this is a helpful tool for any engineering organization that has the same kind of problems Spotify had as in technical sprawl and just visibility of your infrastructure, it's still very important to look sort of self-awareness for the company adopting. I've spoken to so many companies and they do have nuances of differences in what are our challenges. Some companies, their main challenge is how to onboard quicker because for several reasons that may be very technically challenging and you need to jump through a hundred hoops in order to actually onboard. Other are more like Spotify. We had this huge technical sprawl.
I mean, we had the onboarding problem as well, but huge technical sprawl, and we had to focus on how do we drive best practices and standards across our fleet. So I think the first step would be to really decide on which problem to pick, because that's going to help build enthusiasm and understanding for rolling out any developer portal really. So if you identify that problem of yours, then set up some fantastic success metrics that everyone can sort of see, this would be a great thing. Like we did, we had the onboarding metric. Everyone across all layers in the company agreed that that would actually matter, if we could get that down to 20. And after that, then the steps are becoming more and more clear because you have a little bit more clear target because the developer portal clinic can of course be useful in so many different ways, but it's still very good to start with one problem.
Shane Hastie: That organizational self-awareness though is often uncomfortable and again, in my experience, frequently hard to get. How do we, I want to say, overcome the egos, overcome the sensitivity even around having these metrics?
The challenges around metrics [13:43]
Pia Nilsson: I think the challenges I've seen aren't around sensitivity around the metrics, but the sensitivities around already existing products within an organization that then would have to comply with this new idea, this new developer portal that the platform organization want to roll out. That's been the challenge that I've seen in many customer places. What are your thoughts on the metrics? What metrics were you thinking of there?
Shane Hastie: Well, the ultimate metrics for any commercial organization need to be things like employee engagement, customer satisfaction, time to market, profitability. So those have got to be at the top. But then how do we cascade down to something that, I love the example that you're using of employee onboarding time to 10th pull request. Beautiful, because it's very, very explicit and can be clearly defined. To tease out those useful metrics can be, I'm going to say a challenge particularly, and maybe I'm being unfair, but in organizations with a lot of politics. This one I see being difficult, some of these.
Pia Nilsson: Yes, that's very true. I think the way I've seen it successful, and it took us a few years to get this right too, is to really go to the organization within your company, approach these leaders with empathy sort of, and help them co-create metrics that would matter to them. For example, if you want to onboard the security, whatever developed portal to your Backstage instance. So that would be one way of organizationally sort of gluing these pieces together, but also technically just having them integrate with Backstage. I think the way I've seen us do that in a successful manner and also at customer sites is to approach them with respect. So it's very important that these internal customers of Backstage are the champions. They're actually domain expertise and we need to be careful. They are leading their own spaces. I think one of the challenges we are trying to fight often at Spotify is centralization versus decentralization.
So if you have this example of security versus sort of the centralized Backstage team, one has to, instead of saying, we're going to centralize all of it, then you're going to just have to comply with exactly how we would like you to inform your customers, et cetera. One really needs to go at it from the completely different angle, as in how do security want to speak with their customers? How could the Backstage team empower them to do so? What toil do they have today that we can remove so that they want to onboard Backstage?
And this is just a very concrete example, but I think if you sort of apply this approach, then these success metrics for developer effectiveness, they don't tend to become that sensitive any longer because I think the sensitivity is coming from lack of control. So if you steal the control, if I would steal the control from security in this example, certainly I would create a very sensitive topic. They would feel measured, big brotherly treated, et cetera, and then you will really never have real adoption. You will always have to be in this struggle of trying to get their features deployed to the Backstage instance, and it will not sort of fly. Again, they need to want to use this.
Shane Hastie: A term I often use when working with organizations and embracing new ways of working in any area is the concept of invitation over imposition and bringing people along on the journey. And I love what you were saying earlier about that co-creation and people feeling engaged, and that's how people do start to feel empowered. Empowered comes from within, not from without, in my experience.
Pia Nilsson: Yes. I think one has to be pretty passionate about people in order to reach effectiveness and really empathize with, well, what if I was that receiving engineer? Would I have appreciated this? Then the answers become pretty straightforward.
Shane Hastie: So we come back to culture, don't we? That big term that is in many ways nebulous, but is really at the core of every successful organization. I'm going to make that assertion anyway, is the culture that makes people want to be there and to do good work. Spotify is held up as an organization that does have an amazing engineering culture. What do you think puts you on the spot, make Spotify that good? What's the secret sauce? Is there a secret sauce?
What makes the Spotify engineering culture great? [18:35]
Pia Nilsson: I think it has to do with the origin story of Spotify R&D organization. That's at least how I have felt when I joined Spotify 2016. One of the many differences I noticed was how very autonomous people were feeling. That led to a lot of challenges in technical sprawl, blah, blah, blah, all of that. Too much isolated work, but also it was this deep care for sort of your craft and a lot of joy on an everyday basis, which really influenced me. It was a different way of working that I have seen, at least. And while we needed to work on, yes, we need some best practices overall and we need to have a bit fewer differences between the technical stacks on how to, for example, the CI systems, so everyone had their own built Jenkinses or something else of their choice to build their services.
Those were obvious sort of things we need to fix, but that core belief in, well, people are the whole value of this company. I think that has been why it has been fairly easy for Spotify platform organization to invest in bringing product managers along already 2016 when that was very rare actually, to have product folks leading platform tools and development as well as designers, even more rare I would say. So very early on because of this culture that we're building for ourselves and we respect the people at Spotify, working at Spotify. I think that is sort of the secret sauce as I see it.
Shane Hastie: Thank you very much. That's a lovely insight. Pia, there's a lot of ideas that we've explored here together, and thank you very much for your time. If people want to continue the conversation, where do they find you?
Pia Nilsson: Well, they can always reach out to me at Spotify, or they can reach out to the Backstage organization if they are interested in listening in. So backstage.spotify.com. And join our community meetings as well. They would find that on the page to learn more and get in touch with a bunch of Spotifyers and our customers, of course. So there are many ways.
Shane Hastie: Wonderful. Again, thank you so much for your time.
Pia Nilsson: Thanks for having me.
