BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News KubeCon EU: Backstage, Crossplane and Others Preparing for CNCF Graduation

KubeCon EU: Backstage, Crossplane and Others Preparing for CNCF Graduation

In the year when Kubernetes celebrates a decade since its creation, more projects from the CNCF incubated level are preparing to graduate, promising more tools for an ever-widening cloud native ecosystem. The Backstage community has worked on a more robust architecture, and Crossplane aimed to improve its developer experience (DX). KubeFlow and Volcano, both tools promising to improve AI adoption within the Kubernetes ecosystem, are working on easier installation and more features, respectively.

Jorge Castro, open source community manager and developer relations at CNCF, stated at the event that "CNCF incubated projects are the toys for which the community already vouches, as they went through a couple of feedback rounds".

Backstage is an internal developer portal that aims to be a "single pane of glass" that aggregates all the information a developer needs commits, PRs, build pipelines, documentation and others, according to Timbonicus Hansen, staff engineer at Spotify. The project traces its origins to ten years ago when Spotify embarked on a journey to remove the biggest roadblock identified from one of their developer surveys: context switching.

The biggest changes planned currently for Backstage are structural: the team is rearchitecting the system, on both the front and back end, to provide an improved way to build plugins. The clear target is graduating as soon as possible.

Even though the project was the clear winner of the number of end-user contributions to a CNCF project in the last 4 months, the main contributions are coming still from Spotify. An important prerequisite for CNCF graduation is that contributions come from multiple companies. Lately, Red Hat joined the effort too.

Volcano is defined by William Wang, architect at Huawei and contributor at volcano.sh, as the batch tool for the cloud-native era. It is built on top of Kube Edge providing mechanisms like advanced scheduling policies (fair-share, topology scheduling etc.), mechanisms to incorporate SLAs, preempt, and backfill mechanisms.

He also pointed out the importance of batch processing in fields like genomics, machine learning or bioinformatics.

He points to the implementation of job queues from the latest version: "Even if it is a mundane functionality, it is a necessary one when considering batch processing, and it wasn’t available in the Kubernetes ecosystem". According to contributors from both teams, one project that Volcano seamlessly integrates with is KubeFlow.

KubeFlow is a project that aims to "abstract away the infrastructure details in the modern intelligent application era". To be able to give ML teams a way to use all the portability, scalability, and composability of K8s without the hassle of being a Kubernetes expert", according to Chase Christensen, staff solution engineer at TileDB and KubeFlow contributor.

With the help of KubeFlow, teams can pivot between ML frameworks, use Jupyter notebooks, and design hyperparameter tuning jobs, without leaving the framework’s context.

 

Targeted for the mid-year release are the Kubeflow model registry, the new LLM fine-tuning APIs, Kubernetes 1.29 support and pipelines merged into a single GitHub repo. Asked about the feedback from the community in the age of AI he states: "We've also heard a lot of feedback on improving the install experience and are in discussions around improving that as well as conformance for distributions."

CrossPlanes mission is to allow the provisioning and management of cloud resources in a more effective way directly from your Kubernetes cluster. One of the added benefits according to Ezgi Demirel, senior distributed systems engineer at Upbound, is the identification of duplicated provisioning requests that are then throttled, ensuring that exactly the appropriate number of resources is created. Another added benefit is the ability to import by using Terraform schemas (currently neither OpenTofu nor Pullumi is supported).

CrossPlanes mission is to allow the provisioning and management of cloud resources in a more effective way directly from your Kubernetes cluster. One of the added benefits according to Ezgi Demirel, senior distributed systems engineer at Upbound, is the identification of duplicated provisioning requests that are then throttled, ensuring that exactly the appropriate number of resources is created. Another added benefit is the ability to import by using Terraform schemas (currently neither OpenTofu nor Pullumi is supported).

 

When asked about the most important feature of the last version, she pointed to the composition function. These are called by Crossplane to determine what resources it should create when you create a composite resource. The feature is in beta starting with version 1.14.

Demirel also mentioned that when the project applied for graduation, it aimed to consolidate the platform based on the lessons learned so far: Improving DX and adding observability capabilities. In this way, she thinks the project will be better suited for more mature production systems.

One common theme that can be seen in all incubated projects is the push for better usability, developer experience and more robust architectures. As more CNCF projects graduate, ever more capabilities are added to the cloud-native landscape. The goal of the CNCF is to make sure it is prepared for a future full of challenges, from the rapid adoption of AI to the ever-increasing problem of the carbon footprint of cloud infrastructure.

About the Author

Rate this Article

Adoption
Style

BT