The Cloud Native Computing Foundations (CNCF) has recently announced that Kubeflow, the toolkit to deploy machine learning (ML) workflow onto Kubernetes, was accepted as a CNCF incubating project after the vote of the Technical Oversight Committee (TOC).
Kubeflow provides an open-source and Kubernetes-native MLOps platform to develop and deploy distributed machine learning (ML) for the most popular frameworks: TensorFlow, PyTorch, XGBoost, Apache MXNet, and more.
Kubeflow was created by Google in 2017 and now the community counts 150 companies, 28K+ GitHub Stars, 15+ total committers, and 15 releases since 2017. The project is organized into six semi-independent groups:
- The Notebooks Working Group has the responsibility to develop the interface and the interactive deployment environments
- The training Operator group develops the training operators to enable distributed ML training on Kubernetes
- AutoML group develops Katib, an automated model development software
- The Kubeflow Pipeline working group develops the software to convert Python ML scripts into workflow templates
- The Manifest working group develops the installation process
- The KServe project develops high scalable model inference platform on Kubernetes
The current Kubeflow architecture is described by the following image:
Using the Kubeflow configuration interfaces, it is possible to specify the ML tools required for the workflow and it can be deployed to various clouds, local and on-premises platforms both for experimentation and for production.
Ricardo Rocha, TOC sponsor said:
Kubernetes environments provide repeatability, scalability, and fast delivery, making them the perfect place to run AI and ML initiatives. Kubeflow helps fill a gap by delivering machine learning pipelines and MLOps while working closely with its extensive community and other tools and initiatives to create a more cohesive ecosystem. We’re excited to watch the Kubeflow project grow within CNCF and te see the advancements that come in the MLOps space.
The Cloud Native Computing Foundations define three levels of maturity for a project: Sandbox stage, Incubating stage, and Graduation stage.
Every project proposed passes through a process of fallback voting that is described by the TOC graduation criteria:
A two-thirds supermajority is required for a project to be accepted as incubating or graduated. If there is not a supermajority of votes to enter as a graduated project, then any graduated votes are recounted as votes to enter as an incubating project. If there is not a supermajority of votes to enter as an incubating project, then any graduated or incubating votes are recounted as sponsorship to enter as a sandbox project. If there is not enough sponsorship to enter as a sandbox stage project, the project is rejected.
The entrance of Kubeflow as the CNCF incubating project has been celebrated by Taylor D., Cloud Native Computing Foundation (CNCF) head of ecosystem, with a dedicated LinkedIn post.
The main Kubeflow alternative is Amazon Sagemaker, the AWS fully-managed Machine Learning platform.