sqlmesh
dbt-coves
| sqlmesh | dbt-coves | |
|---|---|---|
| 12 | 1 | |
| 1,296 | 209 | |
| 8.7% | 3.8% | |
| 9.9 | 9.2 | |
| 4 days ago | 5 days ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqlmesh
-
Launch HN: Serra (YC S23) – Open-source, Python-based dbt alternative
There is also sqlmesh (https://sqlmesh.com/). Pretty new as well. It introduces some interesting concepts. For smaller dbt projects it could be a drop-in replacement as it allows importing dbt projects.
-
DBT lays off 15% of their staff
I agree with you that they don't have a competitor yet. I think https://sqlmesh.com will be that competitor in the not too distant future though.
- SQL Mesh - Auto DAG generation!!
-
Data transformation tools other than DBT
SQLMesh is a new SQL templating framework that addresses some of dbt's biggest gaps (column lineage, unit testing). It's not an enterprise solution, but it's an interesting project. https://github.com/TobikoData/sqlmesh
-
Semantic Understanding of SQL
It’s a part of the SQLMesh IDE: https://github.com/TobikoData/sqlmesh
- Virtual Data Environments
- Blog Post on how DoorDash used the metrics layer to scale and standardize Metrics for Experimentation
- A dbt killer is born (SQLMesh)
-
SQLMesh: The future of DataOps
If you don't plan on using Airflow, you can just add a custom connection implementation using one of the existing ones as a reference.
dbt-coves
-
Is there something wrong with me, I hate dbt, what am I missing ?
This just feels like you aren’t using the plentiful tools to make those “mind-numbingly slow” dev steps faster. For ex., using dbt-coves to generate the staging models with casting to types in a couple clicks. And pulling directly from Fivetran tables is just poor practice, with the additional steps needed to do it “right” being inconsequential at best.
What are some alternatives?
Mage - 🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai
dbt-ml-preprocessing - A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.
dbd - dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.
magic-the-gathering - A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.
sayn - Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
f1-data-pipeline - F1 Data Pipeline
astro-sdk - Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
datavault4dbt - Scalefree's dbt package for a Data Vault 2.0 implementation congruent to the original Data Vault 2.0 definition by Dan Linstedt including the Staging Area, DV2.0 main entities, PITs and Snapshot Tables.
versatile-data-kit - One framework to develop, deploy and operate data workflows with Python and SQL.
astro - Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow. [Moved to: https://github.com/astronomer/astro-sdk]
serra - Build elegant data pipelines
apollo-server - 🌍 Spec-compliant and production ready JavaScript GraphQL server that lets you develop in a schema-first way. Built for Express, Connect, Hapi, Koa, and more.