So now dbt is worth $4.2b! Yes, that's a "b" for billion.

• dbt-databricks

  15 180 9.5 Python

  A dbt adapter for Databricks.

  

It's pretty straightforward, dbt can use Databricks as the transformation engine, and of course Delta tables can be a source or sink for dbt workflows.

• dbt-spark

  7 364 8.6 Python

  dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks

  

So the idea is you land your data raw in a Delta bronze layer, and then use dbt models to propagate that data forward to silver, gold, do all of your data quality, etc. and all of the actual execution is happening on a Databricks SQL endpoint (or you can use the dbt-spark adapter and run your transfors as Spark on a cluster)

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project