blogpost-airflow-hybrid
Repo that will help you explore how to build a hybrid workflow using Apache Airflow and Amazon ECS Anywhere (by 094459)
Our great sponsors
| blogpost-cdk-mwaa | blogpost-airflow-hybrid | |
|---|---|---|
| 2 | 3 | |
| 9 | 9 | |
| - | - | |
| 3.3 | 0.0 | |
| 4 months ago | almost 2 years ago | |
| Python | Python | |
| MIT License | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
blogpost-cdk-mwaa
Posts with mentions or reviews of blogpost-cdk-mwaa.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-06-20.
-
Upgrading my AWS CDK stacks to AWS CDKv2
In the past when I have talked with builders who do not have Admin access but are using either an IAM user or a Role, they have to following this doc and modify the bootstrap template with the role and permissions they have. (--cloudformation-execution-policies) also see this issue
-
Setting up MWAA to use a KMS key
You can view the complete code in the GitHub repo, blogpost-cdk-mwaa and the changeset here
blogpost-airflow-hybrid
Posts with mentions or reviews of blogpost-airflow-hybrid.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-06-20.
-
Upgrading my AWS CDK stacks to AWS CDKv2
Once the CDK stacks were validated and clear for deployment, I encountered a "feature" when generating IAM policies and roles. It was whilst fixing a different CDK stack (Orchestrating Hybrid Workflows with Apache Airflow, GitHub repo blogpost-airflow-hybrid that I found that even though the stack had synthesised ok, during the deployment, I got the following error:
-
Orchestrating hybrid workflows using Amazon Managed Workflows for Apache Airflow (MWAA)
As always, you can find the code for this walk through in this GitHub repo, blogpost-airflow-hybrid
-
Contributing to the Apache Airflow project
My solution was to use Apache Airflow and create a new workflow to orchestrate this. I planned to create an ETL script, and ensure the script can take parameters, to maximise reuse and flexibility.
What are some alternatives?
When comparing blogpost-cdk-mwaa and blogpost-airflow-hybrid you can also consider the following projects:
amazon-ecs-agent - Amazon Elastic Container Service Agent