Top 23 Airflow Open-Source Projects

• Airflow

  169 34,627 10.0 Python

  Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

  

Project mention: Building in Public: Leveraging Tublian's AI Copilot for My Open Source Contributions | dev.to | 2024-02-12

Contributing to Apache Airflow's open-source project immersed me in collaborative coding. Experienced maintainers rigorously reviewed my contributions, providing constructive feedback. This ongoing dialogue refined the codebase and honed my understanding of best practices.

• argo

  43 14,342 9.8 Go

  Workflow Engine for Kubernetes

  

Project mention: StackStorm – IFTTT for Ops | news.ycombinator.com | 2023-11-05

Like Argo Workflows?

https://github.com/argoproj/argo-workflows

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• incubator-dolphinscheduler

  50 12,041 9.7 Java

  Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

  

• orchest

  44 4,022 4.5 TypeScript

  Build data pipelines, the easy way 🛠️

  

• docker-airflow

  10 3,703 0.0 Shell

  Docker Apache Airflow

• elyra

  2 1,778 6.3 Python

  Elyra extends JupyterLab with an AI centric approach.

  

• airflow-maintenance-dags

  1 1,601 0.0 Python

  A series of DAGs/Workflows to help maintain the operation of Airflow

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• Udacity-Data-Engineering-Projects

  5 1,295 0.0 Python

  Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Project mention: Pitanje za data engineering? | /r/programiranje | 2023-06-30

• dagu

  11 1,174 9.0 Go

  Yet another cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems.

  

• DataEngineeringProject

  5 985 0.0 Python

  Example end to end data engineering project.

• energy-forecasting

  1 792 8.5 Python

  🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

Project mention: Full Stack 7-Steps MLOps Framework – Learn ML and MLOps by Building an ML System | news.ycombinator.com | 2023-09-26

• optimus

  5 737 4.4 Go

  Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management. (by raystack)

  

Project mention: Data Engineering Tools in Go | /r/dataengineering | 2023-05-18

You can check odpf github, they created some dataops tools using go, one of the example is optimus (https://github.com/odpf/optimus) which is a data pipeline orchestrator

• Data-Engineering-Projects

  2 722 10.0 Jupyter Notebook

  Personal Data Engineering Projects

Project mention: Pitanje za data engineering? | /r/programiranje | 2023-06-30

• charts

  3 614 4.7 Shell

  The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes. (by airflow-helm)

  

• streamify

  4 474 0.0 Python

  A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

• astronomer-cosmos

  8 461 9.4 Python

  Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

  

• beginner_de_project

  1 394 2.8 HCL

  Beginner data engineering project - batch edition

• airflow-dbt

  1 379 0.0 Python

  Apache Airflow integration for dbt

  

• Gather-Deployment

  1 351 4.0 Jupyter Notebook

  Gathers Python deployment, infrastructure and practices.

• astro-sdk

  7 319 8.5 Python

  Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

  

Project mention: Orchestration: Thoughts on Dagster, Airflow and Prefect? | /r/dataengineering | 2023-06-01

Have you tried the Astro SDK? https://github.com/astronomer/astro-sdk

• goflow

  1 279 8.4 Go

  Simple but powerful DAG scheduler and dashboard (by fieldryand)

• airflow-chart

  1 267 7.4 Python

  A Helm chart to install Apache Airflow on Kubernetes

  

• airflow-docker

  2 227 3.0 Python

  Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j) (by coder2j)

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Airflow related posts

• Building in Public: Leveraging Tublian's AI Copilot for My Open Source Contributions

  1 project | dev.to | 12 Feb 2024

• Navigating Week Two: Insights and Experiences from My Tublian Internship Journey

  1 project | dev.to | 31 Dec 2023

• Data engineering at home?

  1 project | /r/dataengineering | 10 Dec 2023

• Rockstar Data Engineers making big bucks: what are you doing exactly?

  1 project | /r/dataengineering | 9 Dec 2023

• Airflow VS quix-streams - a user suggested alternative

  2 projects | 7 Dec 2023

• Best ETL Tools And Why To Choose

  1 project | /r/tactionsoftware | 11 Nov 2023

• Simplifying Data Transformation in Redshift: An Approach with DBT and Airflow

  2 projects | dev.to | 7 Nov 2023
• A note from our sponsor - InfluxDB
  www.influxdata.com | 10 May 2024

  Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com