Your default tool for ETL

• dbt-spark

  7 363 8.6 Python

  dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks

  

T: SQL - views and scheduled queries in BigQuery; planning to go hard with dbt as soon as I can find some breathing room)

• airbyte

  139 13,923 10.0 Python

  The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

  

E & L: Airbyte and bespoke Google cloud functions (Python)

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• cargo-crates

  3 1 3.1 Python

  An easy way to build data extractors in Docker.

I went a little crazy and built my own set of data extractors that I can deploy with CDK to ECS.

• damons-data-lake

  4 22 0.0 CSS

  All the code related to building my own data lake

I went a little crazy and built my own set of data extractors that I can deploy with CDK to ECS.

Suggest a related project