cronitor-airflow
magniv-core
cronitor-airflow | magniv-core | |
---|---|---|
1 | 1 | |
2 | 75 | |
- | - | |
10.0 | 0.0 | |
about 2 years ago | over 1 year ago | |
Python | Python | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cronitor-airflow
-
Lessons Learned from Running Apache Airflow at Scale
Is anybody out there doing anything interesting with Airflow monitoring?
At my startup Cronitor we have an Airflow sdk* that makes it pretty easy to provision monitoring for each DAG, but essentially we are only monitoring that a DAG started on time and the total time taken.
* https://github.com/cronitorio/cronitor-airflow
magniv-core
-
Lessons Learned from Running Apache Airflow at Scale
We at magniv.io are building an alternative.
Our core is open source https://github.com/MagnivOrg/magniv-core
We can set you up with our hosted if you would like to poke around!
What are some alternatives?
windmill - Open-source developer platform to turn scripts into workflows and UIs. Fastest workflow engine (5x vs Airflow). Open-source alternative to Airplane and Retool.
dron - What if cron and systemd had a baby?
stepwise - Clojure AWS Step Functions library
ploomber - The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
direktiv - Serverless Container Orchestration
proposals - Temporal proposals
orchest - Build data pipelines, the easy way 🛠️
toil - A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.