sqlmesh
serra
| sqlmesh | serra | |
|---|---|---|
| 12 | 2 | |
| 1,317 | 325 | |
| 8.7% | 0.0% | |
| 9.9 | 9.1 | |
| about 6 hours ago | about 1 month ago | |
| Python | Python | |
| Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqlmesh
-
Launch HN: Serra (YC S23) – Open-source, Python-based dbt alternative
There is also sqlmesh (https://sqlmesh.com/). Pretty new as well. It introduces some interesting concepts. For smaller dbt projects it could be a drop-in replacement as it allows importing dbt projects.
-
DBT lays off 15% of their staff
I agree with you that they don't have a competitor yet. I think https://sqlmesh.com will be that competitor in the not too distant future though.
- SQL Mesh - Auto DAG generation!!
-
Data transformation tools other than DBT
SQLMesh is a new SQL templating framework that addresses some of dbt's biggest gaps (column lineage, unit testing). It's not an enterprise solution, but it's an interesting project. https://github.com/TobikoData/sqlmesh
-
Semantic Understanding of SQL
It’s a part of the SQLMesh IDE: https://github.com/TobikoData/sqlmesh
- Virtual Data Environments
- Blog Post on how DoorDash used the metrics layer to scale and standardize Metrics for Experimentation
- A dbt killer is born (SQLMesh)
-
SQLMesh: The future of DataOps
If you don't plan on using Airflow, you can just add a custom connection implementation using one of the existing ones as a reference.
serra
-
Launch HN: Serra (YC S23) – Open-source, Python-based dbt alternative
The pattern of reading from data sources to a Pandas DataFrame defeats the whole point of using Spark[1]. Maybe it's ok for small tables, but you'll probably run out of memory on large tables.
[1] https://github.com/Serra-Technologies/serra/blob/a7a80c77af5...
What are some alternatives?
Mage - 🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai
apollo-server - 🌍 Spec-compliant and production ready JavaScript GraphQL server that lets you develop in a schema-first way. Built for Express, Connect, Hapi, Koa, and more.
dbd - dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.
dbt-coves - CLI tool for dbt users to simplify creation of staging models (yml and sql) files
sayn - Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
astro-sdk - Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
versatile-data-kit - One framework to develop, deploy and operate data workflows with Python and SQL.
astro - Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow. [Moved to: https://github.com/astronomer/astro-sdk]