BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Making a Case against Kubernetes at Ably

Making a Case against Kubernetes at Ably

This item in japanese

Bookmarks

In a recent article in the Ably Engineering series, Ably's engineer Maik Zumstrull explained why real-time messaging platform Ably does not use Kubernetes.

Kubernetes is, without doubt, a very hot topic and is attracting a lot of interest from developers and companies. It aims to provide a platform to automate deployment, scaling, and operations of containers across clusters of hosts. Kubernetes has become one of the preferred solutions to manage container-based and microservice-oriented platforms.

Ably platform runs on AWS and, although all of their software is deployed in containers, it does not rely on Kubernetes or other runtime layers for their orchestration.

This might come as a surprise to many, given Ably's platform complexity and scalability requirements, and certainly one could find some rationale for it in the fact that Ably began its journey before the Kubernetes' era began. In fact, as Zumstrull explains, using Kubernetes would not provide any significant benefits, while making their system more complex.

Zumstrull goes at great length in his article to describe how Ably deals with resource management, autoscaling, traffic ingress, as well as how things could work in case Ably migrated to Kubernetes. He also explains why benefits specifically brought by Kubernetes, such as multi-cloud and hybrid-cloud readiness, infrastructure as code, and the value of a large community, would not balance Kubernetes costs.

After careful cost-benefit evaluation, it seems that introducing such an enormously expensive component would merely move some of our problems around instead of actually solving them.

InfoQ has taken the chance to speak with Zumstrull to learn more.

InfoQ: Starting with a direct question: do you think the current hype around Kubernetes is unjustified?

Maik Zumstrull: Yes, I think people are overexcited at the moment, but I'd like to stress that we don't think this is primarily a quality issue with Kubernetes. We're aware of a lot of really good work being done on the Kubernetes core services, several adjacent products like MetalLB and Calico, and by SRE teams in various places trying to build user-friendly Managed Kubernetes offerings.

Still, people seem too eager to jump on the hot new thing and introduce Kubernetes without having a good story why they need it and how the considerable investment in deploying Kubernetes will pay off later.

InfoQ: Most of the benefits you describe come from running your production environment on a platform that is highly tailored to your needs. So the question ensues, in which kind of scenarios do you think is it worthwhile to opt for developing your own deployment platform instead of using Kubernetes?

Zumstrull: Almost never! Building your own platform from first principles is a project only organizations with tens of thousands of engineers should try to tackle – and looking at existing examples, even they sometimes just shouldn't.

I wouldn't frame what Ably is doing as "building our own platform". I would describe it as using a thin layer of customization and in-house tooling on top of an existing platform. For us, that existing platform is AWS, for other teams it might be Azure or vSphere or even Kubernetes, and they're all potentially valid choices, depending on the needs of the organization.

One complication with Kubernetes as the platform is that it's not self contained: you'll need an additional platform on which to run the Kubernetes cluster or clusters. So you're either dealing with a cloud provider anyway, or you're trying to run on bare metal and now need a platform for managing physical machines. But depending on what you're trying to achieve as an organization, that might be worth doing.

InfoQ: Could you try to provide an assessment of how complex your deployment platform is, and how long it took for you to create it?

Zumstrull: The initial build of our platform happened before my time at Ably, but I've spoken to people who've been around since day one and they estimate that it took about a year from initial plans to having a running platform that already recognizably resembles what we do today, but there have obviously been many refinements since then.

It's worth noting that this happened before Kubernetes 1.0 was even released, so for Ably, the decision was never whether to build a new venture on Kubernetes or not, but whether it makes sense to retrofit a platform that's already up and running on AWS to Kubernetes after the fact.

If you want to know the full details of how Ably operates its platform without using Kubernetes, do not miss Zumstrull's article.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Dissapointed.

    by Razvan Cosmin,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Although I appreciate other opinions, this articles leads to nowhere.

  • Re: Dissapointed.

    by Daniel Bryant,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I respectfully disagree, Razvan. Highlighting the source article for folks that might have missed it is useful. And from the Q&A I can see how Ably has focused on creating a "minimum viable platform", which is something I'm guessing quite a few organisations could learn from. At the very least, this news post might get platform architects thinking about the Kubernetes decision

    Best wishes,

    Daniel

  • Re: Dissapointed.

    by Sergio De Simone,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thanks for your comment, Razvan.

    In my view, it is always good to learn about alternative approaches to what we take for granted. Kubernetes is a great tech, but its complexity can be daunting. This has made a number of companies stay away from it. Ably is just a case in point, and one that can be useful to critically review architectural decisions based on what others have been experimenting with.

    Best

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT