Tracy Miranda on Secure Supply Chains, SBOMs, and SLSA

Welcome to the InfoQ podcast

Daniel Bryant: Hey, everyone. Before we get into today's podcast, I want to share that InfoQ's International Software Development Conference, Qcon, will be back in San Francisco in the US from October 2nd to 6th. QCon will share real world technical talks from senior software development practitioners. You'll learn about their successes, their failures, and you'll also see how to apply emerging patterns and practices to address some of your challenges too. Learn more at qconsf.com. I'll be there running the platform engineering track, and I hope to see you there.

Introductions [00:25]

Daniel Bryant: Hello and welcome to the InfoQ Podcast. My name is Daniel Bryant, and today I had the pleasure of sitting down with Tracy Miranda to talk about all things secure software supply chain. We look at the actual foundations of this, we look at things like software bill of materials, SBOMs, we also look at SALSA and other acronyms you may have heard in this space. And Tracy goes super deep into what those mean, and how to get started with dealing with secure supply chains.

I've known Tracy for many years now from my work at Chain Guard, from work at CloudBees, and I was super excited to have this conversation. So, welcome to the in InfoQ Podcast, Tracy. Could you introduce yourself to the listeners, please?

Tracy Miranda: Hi, Daniel. Thanks so much for having me. Yes, my name is Tracy Miranda, and Yes, I'm a technology leader and a veteran of open source, so I've worked at a number of different companies, including CloudBees, Chain Guard, the Continuous Delivery Foundation, and the common theme has always been open source.

What do you think is the biggest problem in the security space that developers face today? [01:19]

Daniel Bryant: Fantastic. Now, you and I talked on the podcast officially I think two years ago, and a lot has happened since then, Tracy. What do you think is the biggest problem in the security space that developers face today?

Tracy Miranda: Yes, and it's a super interesting space and I've definitely been drawn into it more and more. And I think the big takeaway is that there isn't one problem, and that's the problem, that tackling security in software is actually solving multiple different problems all at the same time and just doing one won't get you there. So Yes, it's pretty tricky, because it's a space where, if you try to simplify it or underestimate it, you're just not going to get something effective.

Do you think DevOps has helped developers manage security more effectively over the years?  [02:00]

Daniel Bryant: Do you think DevOps and those kind of approaches have helped over the years? Because now we're seeing, what, I think it's 13, 15 years DevOps has been around. Is it making a difference to the collaboration, say, across the teams in regards to security?

Tracy Miranda: I think it's really interesting with DevOps and folks tend to talk a lot about breaking down silos between teams, and you've got that shift left phrase, which I'm not a fan of, but it highlights the fact that things are moving earlier in the pipeline. But security actually has been pretty resistant to a lot of those patterns, it's been super hard to shift left security as such. It's still seen as a very top down type of approach. Security teams and infrastructure teams aren't as well jelled as they need to be to support each other effectively.

I think we're still in the early days, I think the whole DevOps and we've had the DevSecOps iteration, but there's still a long, long way to go on that road. But the good thing is that, we do seem to be heading, as an industry, in the right direction, albeit maybe a bit slower than we need to be.

How has platform engineering impacted software security implementations? [03:13]

Daniel Bryant: No, I think that's IT in general. We are generally going in the right direction, but we like to loop around, and what was it? History doesn't repeat, but it rhymes quite a lot. And when I see some of these things pop up, I'm like, "Oh, we've had that before." But before we dive into some of the supply chain stuff, Tracy, I did want to briefly ask about platform engineering. So, one thing I've seen with platform engineering, and particularly with[ Team Topologies, my buddies Matthew and Manuel, is really force folks to think about decentralized versus centralized ops. So, some responsibilities go onto the platform team, some responsibilities are going to get pushed out, decentralize their developers.

Have you seen any of that emerging? And I'd love to get your thoughts on, is platform engineering a thing in the security community?

Tracy Miranda: Yes. No, absolutely. I think the way people are thinking about platform engineering is the right move forward, and a lot of the companies who are taking security seriously, it tends to be those platform engineers who are the ones looking at things, they're ones saying, "Okay, we are responsible for pulling in various bits of open source and turning it into this platform. How do we make sure that it's going to be secure? How do we make sure it's not going to be compromised by a supply chain attack?" So Yes, I think definitely security platform engineering if you like...

Daniel Bryant: The next level.

Tracy Miranda: Good approach. But Yes, the intersection of those worlds is where some of the most interesting work is happening at the moment.

What’s the most interesting space to focus on with software supply chains? [04:29]

Daniel Bryant: Fantastic. So, you mentioned about supply chains there, Tracy, I'd love to get your thoughts on, is that the most interesting place to focus at the moment? Because we hear a lot about SBOMs, I'm sure we can dive into that later on as well, but Yes, why has the software supply chain caught security attention, at least, of developers at the moment?

Tracy Miranda: Yes, it's really interesting, because for a long time, there's a lot of focus on application security, and you can almost say we did a decent job in different communities, that it became harder and harder to attack applications, and actually the supply chain then became the weakest link. So, you would have people just not really protecting their CICD systems, not treating them in the same way they would with production system. So, this became a real vulnerability, and then there were a couple of really famous attacks, like the SolarWinds compromise as probably one of the big flash points that triggered a lot of saying, "Okay, we need to take this seriously."

And then the more people started looking at it and saying, "Okay, let's investigate the threat models around your CICD, or your supply chain," the more it was like, "Oh, dear. This is bad." And the potential number of things that could happen, it's quite a big list. And we are starting to see the chickens come home to roost on some of the bad practices around securing your CICD pipeline, and also where and how you're getting open source that you're pulling into your products and applications.

Can you explain what SBOMs and SLSA is? [05:54]

Daniel Bryant: Yes, great. So, we do see a lot of mention of software bill of materials, SBOMs, and I know you and I were talking off mic, a lot of folks says SBOMs, SALSA, it's all these acronyms, and I know we all love acronyms, but could you break down some of those things for us and help us actually understand why should I, as a developer, be interested in a software bill of materials?

Tracy Miranda: Yes, absolutely. Like I said, we're still in the early days of trying to figure out what does an ideal secure stack look like. And there's a number of really interesting technologies that are emerging, and each are trying to tackle different spaces. So, one of my favorites in the last few years has been the Sigstore project, which is like the new version of GPG and really gives a fresh developer oriented view to software signing. So, absolutely look at that, Python and PM have all adopted it, safe to say that's going to be one of the building blocks.

And then you mentioned SLSA, and we're seeing that resonate. It's a framework which you can use to think about the different threats and methodically work through the different ways to secure your supply chain, so it's a really nice, focused framework that gives you a starting point and some different levels to work through. So, that's pretty useful. And then, maybe the third S is SBOMs, which is totally really, really interesting. This SBOMs get a lot of buzz, everybody wants to know about them, there's a lot of hype associated with them. And Yes, we could spend a lot of time talking about them, but for those not familiar, it's a software bill of materials, and it's looking to be a format for which you can describe exactly what components go into a piece of software.

And if you think about it, it's a pretty fundamental problem. If you go and talk to companies, and if you go and talk to open source projects and you say, "Can you tell me what software you're running and where you got it from?" You think that would be a pretty basic question. It turns out it's super hard and it's super difficult, and all these big companies and small companies actually don't really know what they're running, and they don't know where it came from, and they don't know whether it's liable to specific vulnerabilities. And the Log4j vulnerability, that highlighted that in a really stark way, that nobody's got this full inventory of what they're running and it's a major, major problem.

I hear the US government is getting involved with software supply chains. Can you share more? [08:13]

Daniel Bryant: I'd love to dive in, perhaps, how folks should get started with that kind of thing, Tracy, but first you and I, off mic, were talking about adoption rates of these things and you mentioned the US government, the Department of Homeland Security, is getting involved in this and it's not often we see government agencies get involved with software engineering, at least not yet. So, I'd love to know a bit more about that. What's the Department of Homeland Security doing around relation to SBOMs?

Tracy Miranda: Yes, so there's a famous executive order, I think it's two or three years ago now, and this was the response to cybersecurity, and it's one of the first executive orders that mentioned open source and I think it mentions it about four times. But part of the crux of it was that, that order really honed in on software bill of materials as a key part of the solution, so the base of the stack. And it tells us a couple of interesting things. One side is that, the US government is really seeing that as a fundamental piece of the puzzle and really is betting that we need innovations and we need developments there, and that's what's going to be the key to solving things, whether or not industry agrees with that.

It has been pretty controversial and there have been mixed reviews to that, but Yes, if I can share a bit more. So, it's not just an order, but the US government has taken it one step further and they're actually conducting a pretty interesting experiment. You could say it's a 1.4 million US dollar experiment. The Department of Homeland Security actually set up an innovation program where invited companies to apply for funding for SBOM projects in open source. So, they're putting their money where their mouth is, and so they said, "Okay, we want people to develop technology and do it in open source, and it's all got to move the state of the art of SBOMs forward."

Now, the really interesting thing that they did with this grant, is that they required every company that applied and would be successful, was obliged to work on a common tool. And specifically, one of the tools is a multi-format SBOM translator. So, it was like, "Okay, we're going to give you 200k to write some tools, but we all want you to work and collaborate on a tool that's going to solve the SBOM format," was this two main formats, and it's a bit of a sticking point at the moment, which format do you even start with?

So Yes, not all these companies are that familiar with open source. There were seven companies awarded things in total, and now they're all going to work together and come together and work on open source tooling. And actually, the first version is already out, it's called Protobom. And Yes, it's aiming to be like the Switzerland of SBOM format, using protocol buffers, and it's a super interesting project to watch. I know Adolfo Veytia and John Speed Meyers, two of my former colleagues, are doing some great work with the community and leading people there. So Yes, definitely one I'm keeping an eye on.

Are government contractors collaborating effectively? [11:09]

Daniel Bryant: Fantastic. As you mentioned about the US government, I certainly have flashbacks to the healthcare.gov debacle. As much as I very much admire President Obama's direction there, there was famous stories at various conferences talked about the challenges of getting all these different teams working together, but you've said so far, the independent folks that have come together around this format, they are actually collaborating, they are making progress.

Tracy Miranda: Yes, so it's interesting to see there is a first working version and it seems to be progressing steadily. So Yes, fingers crossed this could be one that actually pays off and is good for the wider ecosystem. So definitely, Yes, one to watch.

How can listeners contribute to this work? [11:42]

Daniel Bryant: Fantastic. Can interested listeners get involved, Tracy? Can they even just watch what's going on, but can they contribute in any way?

Tracy Miranda: Absolutely. So, the idea is that they do want to build this and open it up in a sustainable way. So, the cohort has put together some community guidelines, they are developing it as open source, and Yes, contributions are welcome. So Yes, I can get you the URL and I'm sure they'd welcome more eyes on it and people getting involved. And I think the URL is pretty fun, it's on github.com, and it's bomsquad, B-O-M.

Daniel Bryant: Nice.

Tracy Miranda: And Protobom is the project itself. So Yes, I encourage folks to check it out and I'm sure they'd welcome help and involvement.

Daniel Bryant: I must confess, I had not heard of that project, so that sounds fantastic. You mentioned in the conversation there about two different formats. I have bumped into that when I've been playing around with SBOMs, with Docker, because I know Docker's got an integrated command. I love Buildpacks, I was messing around with Buildpacks and trying to generate SBOMs, that kind of thing. I did bump into some of this.

Could you briefly outline the two competing SBOM formats? [12:40]

Could you briefly outline for the listeners what the two competing formats are, and is it a VHS Betamax situation? Will there be a clear winner, do you think?

Tracy Miranda: Yes, so the two main ones are SPDX, and folks might know that as the format, I think it's software package data exchange, so it came from the licensing world. And then CycloneDX is the other one. I will add, those are the two main formats, but there's also specific versions of them, and they also have encoding. So, even with those formats, there's a lot of variability. And it's funny, because the last time I was on the podcast, my thing is interoperability and simplifying things. And this is one space we don't want to get that wrong. We think the two different things have different approaches, and some people will say CycloneDX is simpler if you're getting started, but SPDX has a wealth of metadata that allows you to do rich specifications.

But I think if you take a step back, they're 80% very, very similar. And it's not great for people in the industry to come in and try to deal with the inventory problem, try to deal with software supply chain, and they're stuck at the first hurdle. And unfortunately, part of it ends up in a bit of religious format, was something we totally need to avoid. And I think that's why folks are excited about Protobom, because I think it will sidestep a lot of that. That's the hope. Hopefully it doesn't just put out yet another format.

Daniel Bryant: I'm thinking of the XKCD cartoon where there's like, "We've got 17 plus one standards." But no, I think it's fantastic, it's good aspirations. And it's something I'll definitely check out after the call and I'll have a look around. Protobuf is obviously a fantastic spec, so that is a good start to be building from, right?

Tracy Miranda: Yes. And I think it's fair to say, if we think about where we are with SBOMs today and people use the crawl, walk, run analogy, definitely as an industry in a crawl stage. Very few people are producing them. Even if they're producing them, they're hardly spec compliant, and it's a long way to go to making them useful. But that being said, I'm a firm believer that you do want an interoperable spec that everybody can use and subscribe to, so I'm hopeful it can be a good bottom of the stack for software supply chain security.

How can developers get started with creating SBOMs? [14:45]

Daniel Bryant: Yes, fantastic. In my little world with Kubernetes, the YAML specs have actually enabled quite a bit of interesting foundations to build upon. If we can all agree on a foundation of a stanza of a spec, it just means that we can focus on more interesting problems. So, I think it's fantastic. Just dialing it back a little bit, Tracy, as a developer, if we're looking to get started with SBOMs, have you got any recommendations, any sites, books, tools to play around with? How should I, as a developer that's interested in this, how should I start playing around with this stuff?

Tracy Miranda: I think it depends a little on maybe which ecosystem you're in. I think if you're in the Go Ecosystem, you're well set up there, Go's a really lovely language and it provides you a lot of the metadata around building up SBOMs and fun tools, like the Cobil tool will spit you out and SBOM is part of the build, which is really nice. But Yes, in general, I think picking one of the different formats and subscribing to that ecosystem. If you're in the cloud native space, I will say check out the bom tool, which is the tool like Kubernetes uses to produce its SBOM, and I think a few other cloud native tools use that, so I think that's a really good starting point to play around with that.

How can developers get leadership buy-in for implementing SBOMs? [15:51]

Daniel Bryant: Fantastic. And the next question I jump to these days is, it's easy to do bottom up adoption. How do I get that top down buy-in? How do I as a, say, engineering manager, as a leader, get the exec buy-in? Because you need that. For this organization-wide security approach, you need that exec buy-in. I know this is a big question, but have you got any advice on how to get this SBOM adoption and what benefits offers to the leadership, to the business?

Tracy Miranda: One of the things with SBOMs, part of the controversy around it, is that there's a lot of talk of regulations and top-down requirements being mandated, certainly by the US government or even other governments, and that's received a lot of pushback. And I'll say it's limited and it's a bit of a chicken and egg situation. You want to require people to have SBOMs, but the tooling and the standards and the things makes it really difficult.

Yes, but I think just going back to the fundamental problems organizations need to solve, just thinking about the fact you just don't know, you can't run that software inventory. I think highlighting that, that's so fundamental. There's a couple of organizations, I think, who will move forward and certainly the big cloud providers, I think, will start leading the way. Then I'm hoping it'll be a case of using that FOMO just to show what other people are doing with SBOMs and just pushing along and having everybody start producing them.

Can you share a case study about an organization adopting SBOMs? [17:12]

Daniel Bryant: We were talking off mic, Tracy, about a very interesting case study you had with eBay and SBOMs. I'd love to hear a bit more about that, if that's possible, please?

Tracy Miranda: About a year ago, I had the pleasure of speaking to Justin Abrahms at eBay, and just learning more about their SBOM strategy and the practical considerations they were facing. And it was a super fascinating conversation. So, what they shared was, SBOMs are a key part of their wider software supply chain security initiative, and they have a code base which multiple languages, so Java and JavaScript, and JavaScript is always something I have a few more concerns around.

So, a couple of things they shared. One was that formats didn't really matter to them. They were able to generate SBOMs in multiple formats, both SPDX and CycloneDX. And as far as they were concerned, the file sizes were pretty small. And they had two key consumers, so one was the security team, which I think lots of people expect and are familiar with, and they wanted to use the SBOMs for vulnerability management.

But the other one, which I hadn't come across before, was they saw as an internal consumer was their OSPO, so their open source program office. And the OSPO was interested in using the SBOMs to understand which dependencies they were reliant on, and then doing health checks on those, and maybe thinking about, "If we're using this open source project, is it healthy? Should we help sustain it? Should we be contributing?" So Yes, I thought that was a pretty fascinating way to use SBOMs. And in general, they also did a lot of work on SBOM quality, and that was pretty interesting.

What is “SBOM quality”? [18:45]

Daniel Bryant: I'd love to know a bit more about SBOM quality, actually, can you break that down for us?

Tracy Miranda: So, there's a lot of questions about how useful are SBOMs and what is the quality. Unfortunately, in the case of eBay, they found that it was pretty poor, that even the SBOMs, they could find out they weren't even spec compliant. But another aspect of it I find, and this is more general, is that a lot of the SBOMs we have generated today are what I would call guess BOMs.

Daniel Bryant: Oh, no.

Tracy Miranda: Yes. They're generated by scanner tools, which are trying to introspect the final artifact.

Daniel Bryant: Oh, interesting. Yes, yes.

Tracy Miranda: Yes. And trying to come up with complicated rules to say, "I think this is what it's made up for," which is just not really useful and there's a bunch of research on how this just leads to all sorts of confusion, and you might even have WordPress artifacts which don't say they have WordPress in them, or just totally weird things because the scanners can't always pick them up. So, I think one of the things that's going to be important for the industry, is if you want to take SBOM seriously, they really have to come from the base tools themselves, so the build tools, and either by capturing that build information or being able to generate them.

And again, the Go Ecosystem is a good example of an area where a lot of that data is captured and you can make pretty accurate SBOMs for Go. But yes, I think a lot of other ecosystems may need to consider, how do we do that? And as an industry, I think we need to be thinking about more, how do we take a step back and stop trying to generate these guess BOMs and thinking they're going to get us there? But go back to basics and look at our fundamental tools and ecosystems, and work out how we can propagate that information through the compilers perhaps even.

How do SBOMs relate to application dependencies and container/OS libraries? [20:29]

Daniel Bryant: That's our final question. Something rattled around my brain a little bit, because my background is very much Java and then a little bit of Kubernetes and Docker and that kind of thing, what's the interplay there between the language and, say, the packaging format, which are deploy applications? Do the SBOMs take into account both things, because I've got obviously libraries in my Java app and I've got libraries in the OS, on the actual container. I'd love to know a bit more about the interplay there.

Tracy Miranda: Yes, and I think what you're touching on, one of the challenges with SBOMs is all related to naming. So, how do you identify a specific package? Is it a library, is it something else? What's its version? Is it the Log4j package, which has the vulnerabilities or not? And that gets into another can of worms of different formats for software identification, and a couple of different approaches which tackle things differently.

So, while it seems, on the surface, maybe SBOMs should be a simple problem, at the end of the day, it's a big data problem. You've got information about the naming, information about the packages, information about where it came from, and we really need that to be more structured and to be able to be used in a more powerful way. And we're not there yet, but I think I'm hopeful that we can stop operating at a pretty low level and start getting to just more sophisticated tooling and structure. And Yes, just treat SBOMs like the big data problem it is.

Daniel Bryant: You mentioned big data there, Tracy. I've got to ask. Is there any place for AI here? I always hate myself for saying it, but you know what I mean, because you want to be careful, you don't want to probabilistically assume, because you mentioned the guess BOMs. So, LLM's probably out, for example, but do you see any future for AI to be involved in this space?

Tracy Miranda: Yes, if I step back to the wider software supply chain, I think that there are a few places for AI. I think we have to be careful because, in general, AI is not going to be a silver bullet. If you think about, we traditionally have not been good at securing software, so what's all this AI being trained on?

Daniel Bryant: Yes, that's a good point. That's a really good point.

Tracy Miranda: But there's some interesting projects, and I think the Google security team, the GOSST team, they were looking at using AI for open source fuzzing and maybe doing that at scale. And so, that's super fascinating. So, I do think there will be some interesting applications, but Yes, definitely not across the board and there's no silver bullets, magic amplification.

Daniel Bryant: Yes, it makes sense. I definitely heard your message here of foundational and was it the crawl, walk, run, totally makes sense. With something like security, we need to be super thorough in this kind of approach, and AI sometimes is a bit more fun rather than thorough. So, I totally hear you on that. It's been fantastic, Tracy, we've covered the world there in terms of SBOMs, why we should do it, and how we should do it, these things. Is there any final comment there you'd like to make? Anything I've missed at all?

Tracy Miranda: No, I think it's a gray area. I do think software security is a bit of an existential threat, especially in the world of open source. So, I think it's as boring as it might seem, or as painful as it might seem. I do encourage everyone to get involved to start asking questions and to start pushing for the right kind of change in the right direction. And I'm a strong believer that open source communities have a way of coming together and tackling really big problems, so I'm confident that can happen in this case.

Getting in contact with Tracy [23:41]

Daniel Bryant: Superb, Tracy. If folks want to reach out, get involved, chat more with you, where's the best place to find you online?

Tracy Miranda: Yes, just find me either on LinkedIn, I'm Tracy Miranda. Or on Twitter/X, I'm also there too.

Daniel Bryant: Perfect. Thanks for your time today, Tracy.

Tracy Miranda: Thanks. Yes, pleasure to be here. And Yes, love talking about software security.