Top 11 data-mesh Open-Source Projects

• data-contract-template

  1 441 5.0

  Template for a data contract used in a data mesh.

  

Project mention: Data contracts - do you use them? | /r/dataengineering | 2023-05-05

Data contracts can be defined as one of the above, or all of the above. PayPal has recently published their data contract definition here: https://github.com/paypal/data-contract-template

• squirrel-core

  1 279 5.6 Python

  A Python library that enables ML teams to share, load, and transform data in a collaborative, flexible, and efficient way :chestnut:

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Dataplane

  1 183 8.3 Go

  Dataplane is a data platform that makes it easy to construct a data mesh with automated data pipelines and workflows.

  

• data-management-zone

  1 164 2.1 Bicep

  Template to deploy the Data Management Zone of Cloud Scale Analytics (former Enterprise-Scale Analytics). The Data Management Zone provides data governance and management capabilities for the data platform of an organization.

  

• opendatadiscovery-specification

  2 116 6.2

  ODD Specification is a universal open standard for collecting metadata.

  

• data-mesh-demo

  1 59 4.9 Java

  A Data Mesh proof-of-concept built on Confluent Cloud

  

• conduktor-poc-kafka-protocol

  1 58 7.7 Java

  POC to demonstrate how to alter incoming/outgoing records in Kafka. It's a toy, don't use it in production.

  

Project mention: What are your favorite tools or components in the Kafka ecosystem? | /r/apachekafka | 2023-05-31

They also provide an open-source Kafka proxy which can be used to enhance Kafka with 'interceptors'.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• squirrel-datasets-core

  2 43 2.3 Python

  Squirrel dataset hub

  

• trino_data_mesh

  1 8 1.8

  Proof of concept on how to gain insights with Trino across different databases from a distributed data mesh

• ansilo

  1 4 10.0 Rust

  Unlocking the power of SQL/MED to create data ecosystems from disparate data sources

  

• data-mesh-yelp

  1 4 7.1 Smarty

  A complete data product project used to study concepts related to data enginerring such as modeling, collection, operations and so on.

Project mention: You get what you Measure: Understanding your applications health with Grafana, Loki and Prometheus | dev.to | 2024-04-24

Prometheus can be deployed using the Prometheus Helm Chart. This helm chart contains a lot of features such as the already mentioned Push Gateway, Alert Manager and so on. For simplicity reasons of this tutorial I will not show all the Helm chart configuration but you can see a real example used by me here.

data-mesh related posts

• What even is data mesh

  2 projects | news.ycombinator.com | 29 Jul 2021

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com