Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Podcasts Arun Gupta on Managed Container Control Planes on AWS

Arun Gupta on Managed Container Control Planes on AWS


Arun Gupta discusses with Wes Reisz some of the container-focused services that AWS offers, including differentiating ECS and EKS. Arun goes into some detail the role that Amazon Fargate plays and goals behinds EKS. Arun wraps ups discussing some of the open source work that AWS has recently been doing in the container space.

Key Takeaways

  • ECS & EKS are both managed control planes; Amazon Fargate is a technology used to provision clusters.
  • ECR is the Amazon Container registry (similar to the Docker Registry).
  • EKS is an opinionated why of running a Kubernetes cluster on AWS. It is a highly available managed control plane available on US East 1 and US West 2 
  • EKS uses a split account. The control plane runs in an Amazon account and the workers run in customer’s account.
  • Upstream compatibility is a core tenet of EKS.

You moved to AWS a few months after Adrian Cockroft. Is there any connection?

  • 2:00 I have known Adrian for some time - we worked together at Sun.
  • 2:05 We were at Stockholm airport, and I was sitting right next to him on the train.
  • 2:20 He said that he was starting a new team up in 2016 and when he had a req he would let me know - and here we are.
  • 2:35 It was all over very quickly - a week or two from interview to offer - I was very impressed with their HR process.

What does a principal open-source technologist at AWS do?

  • 2:55 Adrian has a team under him, focussed on different verticals.
  • 3:05 As a principal open-source technologist I’m focussed specifically on containers and serverless compute.
  • 3:50 My role is to build strategy around open-source for containers and serverless.
  • 3:50 One of the things I helped do when I joined AWS was to join CNCF (Cloud Native Computing Foundation) where I am Amazon’s board representative alongside Adrian.
  • 11:50 I’m responsible for the CNCF strategy at Amazon - I liaison between the different service teams and different products.

What does a typical day look like?

  • 3:45 My job requires hands-on coding, so I do a lot of coding.
  • 3:50 It could be anything from creating a Docker container to spinning up an EKS cluster to setting up an ISDIO or service mesh.
  • 4:05 A lot of it is reviewing the blogs that our partners, customers or SAs are writing.
  • 4:10 We’re an outward facing team and are on Twitter as well.
  • 4:15 We write abstracts for conferences and submitting presentations.
  • 4:20 I go through the abstract is merit-based approved, rather than sponsored.

What’s the difference between ECS, EKS and Fargate?

  • 5:00 ECS (Elastic Cluster Service) and EKS (Elastic Kubernetes Service) are managed services.
  • 5:10 Fargate is only a technology in how the cluster is provisioned.
  • 5:15 In addition, there’s a managed service called ECR (Elastic Container Registry) - like running a private Docker Hub but in the Amazon cloud.
  • 5:25 Those are the four components that you need to understand.
  • 5:35 ECS was launched in 2014 - there was no container orchestration frameworks, Kubernetes wasn’t open-sourced at that time.
  • 5:45 Our customers were asking to have a service where they could orchestrate containers - and so ECS was launched.
  • 5:55 It is successful today - 100s of millions of containers are alunched every day.
  • 6:00 If a customer is all in, they want a deep integration with Amazon WebServices, CloudWatch, CloudTrail and compliance with HIPPA and ECI compliance.

So if you’re not bringing your orchestration, ECS lets Amazon orchestrate your containers?

  • 6:40 Yes, exactly.
  • 6:45 IF you like Kubernetes API and you want Pods and deployments and replica sets, and you want to ensure you don’t have vendor lock-in - to run in the cloud or on private hosted services, then EKS is a good fit.
  • 7:20 The deployment model and API allows you to run on your own Kubernetes clusters or in the cloud.

Where does Fargate come in?

  • 7:35 With both ECS and EKS you get a managed control plane, but you have to bring your own worker nodes in.
  • 7:45 In some cases, you don’t need to wonder about where the compute comes from or how the cluster is provisioned.
  • 8:50 The idea is that you can provision ECS and bring your own cluster nodes from EC2.
  • 8:10 If you don’t care about how the cluster is provisioned, then that’s what Fargate system does.

How does pricing work with Fargate?

  • 8:45 At Re:Invent 2017 we announced ECS and Fargate integration - that’s available today.
  • 8:55 We also announced plans for integration between EKS and Fargate - we can talk about that later.
  • 9:00 When you run ECS clusters where you bring your own worker nodes, you can do on-demand, spot pricing, reserved instances - the prices could vary depending on what you’re using.
  • 9:15 With Fargate, we give you a defined CPU and memory amount with an initial 60s pricing and then per-second pricing after that.
  • 9:30 If you have a burstable load where you want to do something quickly with containers then Fargate might be good for you.

How is Amazon EKS different from upstream Kubernetes?

  • 10:10 One of the core tenets of EKS is upstream compatibility.
  • 10:20 There’s no inferior branches of Kubernetes.
  • 10:30 When you provision an EKS cluster, you get a managed control plane - in either US-East-1 or US-West-2 - and it will give you a highly available control plane with 3 masters and 3 Etcd nodes.
  • 10:45 We have automatic backup, snapshotting of etcd nodes, automatic scaling.
  • 10:55 It’s a lot of undifferentiated heavy lifting that we’re doing for customers.
  • 11:05 You still have to bring your own nodes to the cluster through ECS and attach them to the cluster.
  • 11:10 You can use kubectl from upstream and use that to deploy your pods, manifests or whatever to the cluster.

Are tools fully supported?

  • 11:35 We take an opinionated approach to the control plane, but you can do anything on your compute nodes.
  • 11:40 The worker runs in the customer account, so you can install anything you want on there - but the control plane runs under Amazon’s account.
  • 11:50 Because it’s a managed control plane, we have a set of plugins that we have enabled and are looking for customer feedback.
  • 12:30 Running Kubernetes and managing the cluster is not for the faint-of-heart.
  • 12:35 For example, should you run a single master or three masters? Should the etcd nodes be colocated with the kubernetes masters? How often should you back up or snapshot your etc containers?
  • 12:50 When you are using these with Amazon, what are the best practices?
  • 13:00 A recent survey from CNCF said that 57% of Kubernetes users run on AWS.
  • 13:10 You still have to understand what the AWS best practices are, which is where EKS comes in.

What does bootstrapping an app with EKS look like?

  • 13:30 You fire up an EKS cluster from the console (or an AWS cli).
  • 13:45 When the master is provisioned you can use a cloud formation template which you can use to bring the worker nodes to attach to the cluster.
  • 13:55 You’re using upstream kubectl to control the cluster.
  • 14:00 Typically if you use kubectl it will have configuration to say how to connect to the cluster.
  • 14:05 In AWS you typically use IAM credentials, so there is an additional dependency for performing the authentication mapping using IAM roles.
  • 14:20 Once you’re authenticated, the same kubectl can be used to connect to the application.

What about limitations?

  • 14:50 There’s work to do with EKS - anything we’ve launched on day 1 is a good start, but we always innovate on our customers’ behalf.
  • 15:00 Given the fact that ECS has been out for over 3 years, it helps to having the deeper integrations like CloudWatch and CloudTrail.
  • 15:10 These are some of the items actively on how we can integrate with them - prioritising on customer feedback.

What is the shared responsibility model?

  • 15:30 The shared responsibility model is where Amazon’s responsibility includes looking after the host operating system, the underlying virtualisation layer and the physical security of the data centre.
  • 15:45 Anything on top of that - the guest operating system, the security groups exposed - is the customer’s responsibility.
  • 16:00 For EKS, the control plane is Amazon’s responsibility.
  • 16:05 If you want to upgrade your cluster, it could be a simple check or it could be a manual process - but we’ll take care of the upgrade for you.
  • 16:10 On the worker side: if you want to upgrade the cluster, then it’s the customer’s responsibility - although we’ll provide scripts to help.

What are the success stories of EKS?

  • 16:30 One of the things I like about Amazon is the customer obsession.
  • 16:35 There were about 60 customers that were part of the limited preview which was announced at Re:Invent.
  • 16:45 GoDaddy was a big success story - they released an article about a few weeks back going all in on EKS.
  • 16:55 Myself and the EKS PM were called in to talk about EKS to GoDaddy.
  • 17:15 A few weeks later we found they went with EKS so we were super excited.
  • 17:35 Go to and the customer success stories are published there.

How does EKS and Fargate work?

  • 17:45 EKS and Fargate is available today - we announced that earlier this year.
  • 18:00 What does Fargate mean in the Kubernetes mean?
  • 18:05 Kubernetes is based around nodes, with node agents (kubelet).
  • 18:10 Pods have affinities to nodes and can be deployed on nodes.
  • 18:15 So nodes are a key part of Kubernetes.
  • 18:20 If you look at Fargate, there is no concept of a node.
  • 18:25 Virtual kubelet is a community effort to build something similar to Fargate.
  • 18:40 Virtual kubelet works with a provider, so we have a Fargate provider upstream.
  • 18:50 The key work has to happen in the kubernetes community.
  • 19:00 They are looking at what nodeless means in a kubernetes environment.
  • 19:15 Amazon wants to have upstream discussions and implementation rather than doing things internally.

What are some of the other stories you have at Amazon?

  • 19:30 One of the key tenets of EKS is open-source.
  • 19:35 If you look at how the networking is done with EKS then it’s done with the cni plugin.
  • 19:45 There are about 8-10 cni plugins from various vendors.
  • 19:50 What we did is to build a cni plugin in open-source, and anyone can use it to set up their own cluster with native vpn networking.
  • 20:05 The authenticator we were talking about earlier, to provide IAM credentials to a Kubernetes cluster, was done in open source.
  • 20:15 We talked about virtual kubelet - there is ckws which we are quite active.
  • 20:30 There is CoreOS ingress controller that can be installed with EKS - hopefully sometime in the future we’ll have a prebundled user of CoreOS.
  • 21:00 Last but not least there is a cloud provider working group where we have been working with others to have standard cloud APIs.
  • 21:15 We’re actively hiring for open source kuberentes work.

What’s the story around the service mesh using EKS?

  • 21:30 Because you’re managing the control plane, we define what goes in that based on customer feedback.
  • 21:35 Because you are managing the worker nodes, you can do what you want on them.
  • 21:40 Isdio can be installed on an EKS cluster and will work.
  • 21:50 I was working with a developer advocate earlier on installing isdio on an EKS cluster, to find out what works and what doesn’t, and were filing issues in the open-source as well as making sure it works out of the box on EKS.

What do you have planned for the rest of the year?

  • 22:20 Next week I’m going to be at LinuxCon, ContainerCon, LC3 in Beijing, and an event in Korea.
  • 22:20 Re:Invent planning has already started, we are looking at being at QCon Seattle, meeting a lot of customers around the world and highlighting customer stories.
  • 22:50 I’m particularly passionate about open-source, so if any customers, developers or partners want to blog about open-source on our open-source blog, I’m happy to take feedback.

What are you working on?

  • 23:05 I’ve been interested in service mesh.
  • 23:15 It’s difficult to explain the concept of a pod to a new kubernetes user, but the moment you have the concept of a sidecar automatically injected in the pod, there is an aha! moment.
  • 23:30 Envoy is one part of it, and Isdio, and take that a step forward - that’s what I’m looking at.
  • 23:45 I’m looking at chaos engineering to make your systems more resilient, and how you can build chaos engineering, Isdio and kubernetes together to build resilient engineering.

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article


Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • typo

    by Rob Fulwell,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    "core tenant" should be "core tenet"

  • Re: typo

    by Wesley Reisz,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thanks... fixed.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p