Our great sponsors
-
astronomer-cosmos
Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Well, I would take a look at Astronomer Cosmos which is a framework to dynamically Airflow DAGs from dbt with a single operator.No need to manage connections, use true BashOperator etc. It’s dead simple and open source https://astronomer.github.io/astronomer-cosmos/ I’m going to make a video about it very soon.
Dagster seemed really cool when I looked into it as an alternative to airflow. I especially like the software defined assets and built-in lineage which I haven't seen in any other tool. However it seems it does not support RBAC which is a pretty big issue if you want a self-service type of architecture, see https://github.com/dagster-io/dagster/issues/2219. It does seem like it's available in their hosted version, but I wanted to run it myself on k8s.