SHSelfHostIndex
AP

Apache Airflow

Programmatically author, schedule and monitor workflows

Automation & Workflows ★ 46.4k stars Hard setup Apache-2.0

Apache Airflow is a platform to author, schedule and monitor workflows as directed acyclic graphs of tasks. It is widely used for data engineering pipelines and complex job orchestration.

Official siteSource code

Key features

  • Workflows defined as Python code
  • Rich scheduling and dependency management
  • Extensive operator and provider ecosystem
  • Web UI for monitoring DAGs

Pros & cons

Strengths

  • Industry standard for data pipelines
  • Huge ecosystem

Trade-offs

  • Heavy to operate
  • Steep learning curve

Apache Airflow replaces

Self-hosted AWS Step Functions alternativesSelf-hosted Azure Data Factory alternatives

Compare Apache Airflow

71 head-to-head comparisons.

Similar automation & workflows apps