blogpost-airflow-hybrid
Repo that will help you explore how to build a hybrid workflow using Apache Airflow and Amazon ECS Anywhere (by 094459)
airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows (by 094459)
| blogpost-airflow-hybrid | airflow | |
|---|---|---|
| 3 | 1 | |
| 9 | 0 | |
| - | - | |
| 0.0 | 0.0 | |
| almost 2 years ago | about 2 years ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
blogpost-airflow-hybrid
Posts with mentions or reviews of blogpost-airflow-hybrid.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-06-20.
-
Upgrading my AWS CDK stacks to AWS CDKv2
Once the CDK stacks were validated and clear for deployment, I encountered a "feature" when generating IAM policies and roles. It was whilst fixing a different CDK stack (Orchestrating Hybrid Workflows with Apache Airflow, GitHub repo blogpost-airflow-hybrid that I found that even though the stack had synthesised ok, during the deployment, I got the following error:
-
Orchestrating hybrid workflows using Amazon Managed Workflows for Apache Airflow (MWAA)
As always, you can find the code for this walk through in this GitHub repo, blogpost-airflow-hybrid
-
Contributing to the Apache Airflow project
My solution was to use Apache Airflow and create a new workflow to orchestrate this. I planned to create an ETL script, and ensure the script can take parameters, to maximise reuse and flexibility.
airflow
Posts with mentions or reviews of airflow.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-02-10.
-
Contributing to the Apache Airflow project
Forking Apache Airflow is simple enough, as GitHub makes this as simple as a couple of clicks. I now have my own Apache Airflow fork for for this work. The only thing I needed to make sure was that it was the right branch (latest, main) and not any of the other branches (I have made that mistake before!)
What are some alternatives?
When comparing blogpost-airflow-hybrid and airflow you can also consider the following projects:
amazon-ecs-agent - Amazon Elastic Container Service Agent
Docker Compose - Define and run multi-container applications with Docker
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows