trino_data_mesh
dbt-spotify-analytics
| trino_data_mesh | dbt-spotify-analytics | |
|---|---|---|
| 1 | 1 | |
| 8 | 120 | |
| - | - | |
| 1.8 | 0.0 | |
| almost 3 years ago | almost 2 years ago | |
| Python | ||
| - | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
trino_data_mesh
-
What even is data mesh
Not central to the main ideas of this article, but if you want to have a data mesh that is self-service, why force folks to use a particular storage medium like a data warehouse? That still requires centralization of the data.
Why not instead have a tool like Trino (https://trino.io) that allows you to let different domains use whatever datastore they happen to use. You still would need to enforce schema, but this can be done in tools like schema registry as mentioned in the article along with a data cataloging tool.
These tools facilitate the distributed nature of the problem nicely and encourage healthy standards to be discussed and the formalized in schema definitions and catalogs that remove the ambiguity of discourse and documentation.
Nice example is laid out in this repo of how Trino can accomplish data mesh principles 1 and 3 (https://github.com/findinpath/trino_data_mesh).
dbt-spotify-analytics
What are some alternatives?
soda-sql - Data profiling, testing, and monitoring for SQL accessible data.
frappe - Low code web framework for real world applications, in Python and Javascript
lightdash - Self-serve BI to 10x your data team ⚡️
wal-e - Continuous Archiving for Postgres
Trino - Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
Skytrax-Data-Warehouse - A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for cloud data warehouse and Metabase to serve the needs of data visualizations such as analytical dashboards.
dbt-customer-journey-analysis - Using DBT for Customer Journey Analysis on RudderStack - an open-source, warehouse-first customer data pipeline and Segment alternative.
airflow-dbt - Apache Airflow integration for dbt
dbt-sessionization - Using DBT for Creating Session Abstractions on RudderStack - an open-source, warehouse-first customer data pipeline and Segment alternative.
spotify-data-reader
PostHog - 🦔 PostHog provides open-source product analytics, session recording, feature flagging and A/B testing that you can self-host.