BT

AirFlow Joins Apache Incubator

| by Alex Giamas Follow 10 Followers on Apr 29, 2016. Estimated reading time: less than one minute |

AirFlow recently joined the Apache Incubator program. AirFlow is a workflow and scheduling system designed to manage data pipelines. Developed by AirBnb for their internal usage, it was open sourced last September, as previously reported by InfoQ.

The project has expanded outside Airbnb’s usage to the industry, adding 2 new companies using it every week, around 10 new contributors and 20 new users. At the moment with 30 companies using it in production, 8 committers and 135 contributors, it has a solid base for expansion.

Stacking up against Azkaban, Luigui and Oozie, Airflow has the key differentiator that pipelines are defined as code and tasks are instantiated dynamically. Ease to setup and being developer friendly are also among the design considerations for Airflow’s development.

Joining the incubation program, Airflow is on its way to enter the Apache Software Foundation (ASF) list of projects, along with projects like Impala, DataFu, AsterixDB, Zeppelin and others. Airflow’s code is available at Github and the project is always looking for new contributors.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss
BT