Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Podcasts Gareth Rushgrove on Kubernetes as a Platform, Applications, and Security

Gareth Rushgrove on Kubernetes as a Platform, Applications, and Security

In this podcast, Daniel Bryant sat down with Gareth Rushgrove, Director of Product Management at Snyk. Topics covered included Kubernetes as a platform, application abstractions, continuous delivery, and implementing good security practices in the cloud native space.

Key Takeaways

  • The value provided by Kubernetes depends on an organisation’s context. Kubernetes acts as both a series of lower-level building blocks for a platform, and also as a very powerful API for deploying and operating container-based applications.
  • Kubernetes provides several useful abstractions for engineers. For example, Pods, Deployments, and Services. However, Kubernetes doesn’t have an “application”-focused abstraction. Tools such as Helm and specifications like the Cloud Native Application Bundle (CNAB) are driving innovation in this space.
  • There is a large amount of open source Kubernetes tooling. This has been created by a range of vendors, groups, and individuals. Encouraging this diverse mix of participation is beneficial for the long-term health of the ecosystem. 
  • The Cloud Native Computing Foundation (CNCF) provides a space for people to collaborate regardless of their current organisational affiliations.
  • Defining appropriate standards within the cloud native space is useful for enabling interoperability and providing common foundations for others to innovate on top of. 
  • Security challenges within IT are socio-technical. Security teams working with cloud native technologies will benefit from continual learning, developing new skills, and researching new tools. For example, the defaults of Kubernetes aren’t necessarily secure, but this can be readily addressed with appropriate configuration.

Show Notes

Can you introduce yourself?

  • 01:00 I'm director of product at Snyk; we do security tooling for developers covering open-source components and containers.

Are there core capabilities of a platform based on Kubernetes?

  • 01:45 One of the things with Kubernetes that surprises a lot of people is that it is both a low-level abstraction and a higher-level abstraction.
  • 02:00 If you are thinking of Kubernetes as your future application platform, and it turns out that Kubernetes has got low-level building blocks, then you have to build something on top of it.
  • 02:10 On the other hand, if you are coming at it from a migration path of a bunch of VMs, the Kubernetes API is incredibly high level.
  • 02:30 It depends on where you are coming from.
  • 02:35 If you buy into the idea that you're building a platform on top of it, what is missing in the Kubernetes API is the application.
  • 02:45 You've got Services, you've got Deployments (which adds an abstraction layer over replication sets and pods) - but how do you organise teams, source code, CI around?
  • 03:10 Kubernetes doesn't have that built in, but there has been moves in that direction for a while.

What's the state of Kubernetes tooling?

  • 03:30 There's a lot of opportunity; that was what attracted me to Kubernetes 4-5 years ago.
  • 03:35 The first talk I did at KubeCon was called "Kubernetes and the potential for higher level tools" - and I still fundamentally believe that will happen.
  • 03:45 We haven't moved towards it as quickly as I thought we might, and to a certain degree that's also down to wide-scale adoption.
  • 04:00 It's not just the high level high context people who were interested in Kubernetes because of its API surface; lots of people have been using Kubernetes to get work done.
  • 04:10 Because that potential was there, you do see some of those tools; some have come and gone, some were too early.
  • 04:20 Most people haven't needed the high level tools yet, and that's led to a certain degree of fragmentation.
  • 04:30 I think what's happened was a lot of people jumped on to Kubernetes too early, before the tooling existed, and built their own tools.
  • 04:40 Kubernetes was API-first, and lots of people built their own tooling, which was relatively straightforward to do; you'll see some of that consolidating over time.
  • 04:50 KubeCon is a great proxy for that; in Barcelona, something like 80% of attendees were new to the event, and in San Diego, something like 70% were new to the platform.
  • 05:05 It's easy to forget that most people are new to Kubernetes.
  • 05:10 In that context, if you come to the platform and find the array of tools bewildering, that can be a problem.
  • 05:15 I created a spreadsheet a while back, partly as a narrative of what tools were built on top.
  • 05:30 I don't think too many tools have been created; the ones that see large-scale adoption will win, rather than the ones with the most interesting functionality.

What's your thoughts between pure open source and CNCF as well as vendors?

  • 06:00 I think they can be quite individual and come from a different point of view.
  • 06:05 Some of it comes from what you think of open source, and whether you think open source is important.
  • 06:10 I came back from FOSDEM a few weeks ago, and there were lots of conversations about what open source is.
  • 06:20 When I got into open source, it was about the freedoms for me; we've had conversations about licences and maybe open source has become synonymous with free, but we're coming back to licenses again.
  • 06:40 I think you'll have people coming from different points of view.
  • 06:45 I believe open source is important.
  • 06:50 I don't like the idea of a world where all the open source software that becomes popular and sticky is spat out of large corporate organisations.
  • 07:00 It adds value to the ecosystem; but I like it where there's a mix.
  • 07:05 I've learnt that building big software takes a lot of people, and gathering that amount of people together for free take a long time and is hard.
  • 07:15 Doing it commercially is easier; it doesn't mean it's easy.
  • 07:20 I like the mix that you can go out on your own and create software, or with a small group of people.
  • 07:25 You can see open source projects coming from start-ups, which might succeed or fail, and from end users.
  • 07:40 The interplay is interesting, and I think that relationship is all about the end-user first.
  • 07:50 If you don't focus on the end user first, you're not going to succeed.
  • 07:55 If you build something for yourself, you are the end user - it's really easy.
  • 08:00 If you're building for someone else, whether it's for a startup or a giant corporation, if you're not putting the end user first then over time it won't work out.
  • 08:10 If you have a lot of money, you can get somewhere quite quickly, and you can squash things, but on balance it might not work out.
  • 08:15 Foundations have a different role to play, and work differently - CNCF is central to quite a lot of things in the cloud native space.
  • 08:30 People like the idea that the foundation is making lots of decisions, but actually what's happening is it's lots of people in lots of projects that are making decisions.
  • 08:35 The CNCF provides a place where lots of the technology people can talk, even though they might be from competing companies.
  • 08:45 That's the benefit of the foundation.
  • 08:50 The other aspect is it has driven awareness of cloud native applications and infrastructure, and that shift across lots of companies (including small ones) is a benefit to everyone.
  • 09:10 Large companies, which might not listen to individual users, can be associated with a group to move towards automation and devops - all of those things, we can agree on under one banner.
  • 09:45 When we get down to the specifics of how to offer a specific thing, we can offer competing solutions.
  • 09:50 That's the key benefit of a foundation; people may look down on marketing, but actually it turns out it's really powerful when done well.
  • 10:00 The foundation is also a place where people who are employed by fiercely competing companies can talk about things together, even though they work for different companies.
  • 10:10 Technically it just needs a common project, but the foundation provides things and opportunities that wouldn't exist otherwise.

What is the role of standards in this?

  • 10:30 Standards get a bad reputation, and people run away - I don't think that's going to change.
  • 10:40 I would like to see more work done around standardisation.
  • 10:45 What we're seeing is people trying to build de-facto standards.
  • 10:50 Where that works, it works really well; the projects around the cloud native space are generally to do with containers; securing them, managing them, building them, using them as an abstraction.
  • 11:15 That fundamentally happened down to a widely adopted implementation in Docker, but the smart move in standardising the low-level bits with the OCI, the runtime image formats, etc.
  • 11:40 The number of companies with container registries expands every few months, and the reason why it's not a problem for users is because they're all fundamentally based on the distribution spec.
  • 11:50 It turns out that they aren't 100% compatible; there's always things that people will do either on purpose or not on purpose, but for the most part they are transferrable between registries and runtimes.
  • 12:10 Google cloud run enables you to run containers here or there, hypo was doing that before perhaps ahead of its time; and all of these were based on standards.
  • 12:15 You don't want too many standards, and you don't want to create standards first and then hit people over the head with them.
  • 12:25 I do think specification standards we can all agree on common undifferentiated things is a good space.
  • 12:30 The serverless working group around Kubernetes around the cloud event spec have been doing things there.
  • 12:45 They've been finding areas in which they can agree, and writing those down as lightweight standardisation processes.
  • 12:55 I was recently involved in the Cloud Native Application Bundles (CNAB) spec at the beginning, with Microsoft, Docker, Pivotal etc. about the higher level packaging.
  • 13:05 A lot of that doesn't have to be end-user facing, either; there's a lot of people who are using Docker without knowing about OCI - but they get the benefits of that.
  • 13:20 There are a lot of people wondering how they might use it, but instead, should be wondering which products to use that might use it.
  • 13:25 Most people don't need or want to care how it does it; they care about the products solving the end user problem, which is the way it should be.

What are your thoughts on the good and bad of buildpacks?

  • 14:05 Software packaging and container builds are important.
  • 14:10 One of the things Docker brought it at the right time and right place was to move software packaging from a specialised part of the IT department to self-service.
  • 14:40 Docker packaging shifted really rapidly to developer teams doing packaging all the time.
  • 14:50 There are tradeoffs about how that happened; ultimately with docker build as the tool and the OCI specification as the result.
  • 14:55 One of the things you can't argue about is how widely adopted and simple it was to make that transition.
  • 15:05 IF you talk to the Debian packaging people, they'll list off a whole set of things that are wrong with the Docker files, and they have a point.
  • 15:15 There are several million Docker files on GitHub - it really helped to democratise packaging.
  • 15:30 There are tradeoffs; so the question is, can we have other types of build tools?
  • 15:40 Jib is a specialised tool from Google for the Java ecosystem.
  • 15:50 If you can remove a lot of the general purpose nature abstraction; as long as you fit into this gap, and the tool will take care of the rest of the complexity for you.
  • 16:10 I think buildpacks have ultimately been of interest for those that have already been using CloudFoundry.
  • 16:15 As a direct end-user tool, I think it has some downsides; where I see much more interest in buildpacks is being built in to something they are already using.
  • 16:25 Buildpacks aren't the end user interface; they're an implementation detail - so I really like the recent Spring integration.
  • 16:40 If it gets built into the latest Spring projects, it will automatically build your Spring project into a container using buildpacks.
  • 16:50 You don't need lots of things being able to do that; you just need something that has encapsulated all of those problems.
  • 17:00 There's other build work going on as well; the evolution of 'docker build' into a library called buildkit; lots of software is now using buildkit to do builds without needing docker.
  • 17:15 That's then plugged back into docker as the next generation of the build process.
  • 17:20 There's still some integration required for Windows containers, but there's already a lot of improvements about syntax and extending the Dockerfile syntax.
  • 17:30 For me, the Dockerfile is a great blunt tool for getting to the 80% goal very quickly with a low barrier to entry.
  • 17:40 I think we'll see tools that take a different approach; single tools around specific ecosystems like buildpacks which generate an OCI image.

What do you think a best practice CI/CD pipeline looks like for Kubernetes?

  • 18:10 I think the reality is that it is based on your organisation context.
  • 18:15 There are multiple projects in the CNCF and the CDF from projects and end-users to large vendor-backed projects.
  • 18:30 What it means is that there's a richness to the tooling, but everything is moving really quickly
  • 18:35 It's probably also why I would wager that Jenkins is the tool that everyone's using for doing their builds.
  • 18:45 Everyone's using it, and can make it do what they want.
  • 18:50 In a greenfield environment with a lot of Kubernetes experts, the answer is going to be different from that from one in which there are SaaS solutions.
  • 19:05 I'm interested in Tekton [] (a project stewarded by the CDF), built by Google, RedHat, and other folks.
  • 19:25 What it's doing is providing the elements for software build on top of Kubernetes.
  • 19:30 What it adds to Kubernetes is pipelines, tasks, as constructs in the same way you'd have deployments, pods and services.
  • 19:40 For me, that's serving an interesting problem; at that point, you could have interoperable tools on top which are using those details.
  • 19:55 I think that might be how we get to genuinely better tooling.
  • 20:00 There are other tools available; Argo and ArgoFlux is taking a slightly different approach.
  • 20:10 I would love to see other things built on top of Tekton, because the interoperable bit appeals to my nature.
  • 20:15 The Kubernetes-native objects is a useful API abstraction.
  • 20:20 It's a powerful abstraction, but there's a high barrier to entry.
  • 20:35 Viewing these as as end-user tools is probably the wrong level.
  • 20:40 Viewing the tools that use these things is probably the correct way of doing it.
  • 20:45 If people feel that Kubernetes is complex - but there are a lot of people who are using Kubernetes without attending KubeCon - they're not using them and they don't find it complicated.
  • 21:05 The moment you add custom resource definitions and operators, the complexity goes through the roof.
  • 21:10 I feel that they're an implementation detail of what can be done, but not an end user visible thing.

What do you think are the biggest security challenges with Kubernetes?

  • 21:40 Security problems are always socio-technological; they're always a mixture of both people and technology.
  • 21:45 It's nearly impossible to say which one is more important, because it will depend on the actual problem at the time.
  • 21:55 We talked about previously, about the shift of software packaging to something that developers own (and associated problems).
  • 22:10 We are seeing it move from a smaller number of people who bear all the responsibility and assume all the load and act as a gate to good or bad is breaking down.
  • 22:20 Security is something that is shifting towards being a developer problem.
  • 22:25 That isn't to say - in the same way as software packages - that there aren't expert people who know how to do the high-end bits.
  • 22:30 Security definitely needs security experts - with the shift to devops, the sys admins didn't disappear, they became SREs.
  • 22:50 You have seen the value and appreciation of SREs grow over time - security is going to go through the same existential crisis.
  • 23:00 The security people are going to become the new security experts, and instead of playing whack-a-mole with thousands of things and focus on the high-risk, high-security problems.
  • 23:15 Kubernetes is happening in the space where there's a lot of things shifting; packaging, operations, configuration, security etc. is being pushed down to developers.
  • 23:25 So what are the security tools and problems that are fundamentally application security problems, but they spill over into historically what would have been IT and infrastructure space.
  • 23:40 The line of IT owning databases and load balancers and the devs owning the applications is breaking down; the developers are bringing their own databases and owning more of the data.
  • 24:00 It's in that context that all of this is happening; developers need help to understand what is happening, but security needs new tools and to understand the evolving space.
  • 24:15 The fundamentals, like ephemeral API driven infrastructure, is the new normal, and it's not sufficient to (say) to get a list of all of the IP addresses?
  • 24:30 The problem is that the spreadsheet will be out of data before it's even used, so is it useful?
  • 24:35 So it moves the discussion to security fundamentals - one of the main things is to go back to basics, and do a thread modelling exercise.
  • 24:55 You'll then discover the problems as you go through that process.
  • 25:10 There's a CNCF SIG about security, and there were a security days at KubeCon which was about a cross-section of end users.
  • 25:30 The work that's going through that community between all stakeholders and security people is important.

If I'm new to the security space, are there any quick wins?

  • 26:10 One is understanding that Kubernetes' defaults aren't that secure.
  • 26:15 That's something the Kubernetes project acknowledges, and it isn't as much a mistake as much as hindsight of massive success.
  • 26:20 If you understand the defaults, like things are running by default as privileged or with writable file systems; if you can avoid those, you can remove a lot of attack vectors.
  • 26:40 Another is that a pod can use as much as it wants to on a node, and the other things get thrown off; you end up with risks around resource starvation.
  • 27:00 Adding limits can mitigate some of those problems.
  • 27:05 You need to address the secure defaults on a per-application basis.
  • 27:15 The biggest problem aside from people deploying Kubernetes control planes to the internet (don't do that!) is the security of the application itself.
  • 27:30 People might not attack Kubernetes, but if the applications have SQL injection problems or cross-site scripting problems, the most secure Kubernetes platform won't help you.
  • 27:55 You don't win by protecting the data or compute if you don't also protect the application you put onto the platform.
  • 28:00 Ultimately thinking about the security of the applications and vulnerabilities they may have are the best places to start.
  • 28:20 It doesn't require you to buy into a lot of tooling to make it happen.
  • 28:30 If you are in a large organisation, and have intrusion detection systems or business intelligence solutions you can get your data in to, you should use those.
  • 28:35 You still need a level of expertise to use the data and find the signals.
  • 28:45 There are things on the developer side that you can scale out faster to cut down the signal in order to help.
  • 28:55 Bang for buck, secure your applications and understand that Kubernetes isn't secure by default are the two main ones.
  • 29:00 Ian Coldwater did a talk on "Think like an attacker
  • 29:15 Ian also did a talk from Duffy Cooley from VMWare on the problems of a security point of view of Kubernetes defaults 

What is the best way for people to follow you on-line?

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article