CI/CD in data engineering - help a noob

• dbt-fal

  12 851 7.7 Python

  Discontinued do more with dbt. dbt-fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.

  

Thanks! I am resisting the urge to add airflow to my project, not sure for how long. I have a setup without an orchestrator (airbyte + dbt). So far I am able to shoehorn everything into my setup using airbyte scheduled syncs and invoking dbt directly from there. We have our in house tool that we use for scripts (datascience workflows) we run after dbt.

• dbt-expectations

  10 939 6.7 Shell

  Port(ish) of Great Expectations to dbt test macros

  

There are certain things I would like to add such as data quality, I can use something like dbt great expectations, but I am not sure how much more I should force it before getting an airflow setup..

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project