BT

Facilitating the spread of knowledge and innovation in professional software development

Contribute

Topics

Choose your language

InfoQ Homepage News AirFlow Joins Apache Incubator

AirFlow Joins Apache Incubator

Bookmarks

AirFlow recently joined the Apache Incubator program. AirFlow is a workflow and scheduling system designed to manage data pipelines. Developed by AirBnb for their internal usage, it was open sourced last September, as previously reported by InfoQ.

The project has expanded outside Airbnb’s usage to the industry, adding 2 new companies using it every week, around 10 new contributors and 20 new users. At the moment with 30 companies using it in production, 8 committers and 135 contributors, it has a solid base for expansion.

Stacking up against Azkaban, Luigui and Oozie, Airflow has the key differentiator that pipelines are defined as code and tasks are instantiated dynamically. Ease to setup and being developer friendly are also among the design considerations for Airflow’s development.

Joining the incubation program, Airflow is on its way to enter the Apache Software Foundation (ASF) list of projects, along with projects like Impala, DataFu, AsterixDB, Zeppelin and others. Airflow’s code is available at Github and the project is always looking for new contributors.

We need your feedback

How might we improve InfoQ for you

Thank you for being an InfoQ reader.

Each year, we seek feedback from our readers to help us improve InfoQ. Would you mind spending 2 minutes to share your feedback in our short survey? Your feedback will directly help us continually evolve how we support you.

Take the Survey

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT

Is your profile up-to-date? Please take a moment to review and update.

Note: If updating/changing your email, a validation request will be sent

Company name:
Company role:
Company size:
Country/Zone:
State/Province/Region:
You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.