Apache Airflow!

Airflow is a platform to programmaticaly author, schedule and monitor data pipelines.

Go to http://airbnb.io/projects/airflow/

Apache Airflow alternatives

RunDeck

RunDeck is an open source automation service with a web console, command line tools and a WebAPI. It lets you easily run automation tasks across a ...

tags: automation server-management system-administration deployment configuration
platform: Linux

Apache Oozie

Oozie is a workflow scheduler system to manage Apache Hadoop jobs.

tags: jobs hadoop workflow-tool
platform: Linux

Azkaban

Azkaban is a batch workflow job scheduler created at LinkedIn to run Hadoop jobs. Azkaban resolves the ordering through job dependencies and provid...

tags: workflow jobs etl hadoop
platform: Linux

Luigi

The purpose of Luigi is to address all the plumbing typically associated with long-running batch processes. You want to chain many tasks, automate ...

tags: etl
platform: Linux