Top 10 data-mesh Open-Source Projects

1. data-contract-template

   1 1 469 5.0

   Template for a data contract used in a data mesh.

   

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. squirrel-core

   2 1 281 3.6 Python

   A Python library that enables ML teams to share, load, and transform data in a collaborative, flexible, and efficient way :chestnut:

   

4. Dataplane

   3 1 226 5.4 Go

   Dataplane is a data platform that makes it easy to construct a data mesh with automated data pipelines and workflows.

   

5. data-management-zone

   4 1 171 2.1 Bicep

   Template to deploy the Data Management Zone of Cloud Scale Analytics (former Enterprise-Scale Analytics). The Data Management Zone provides data governance and management capabilities for the data platform of an organization.

   

6. opendatadiscovery-specification

   5 2 135 4.6

   ODD Specification is a universal open standard for collecting metadata.

   

7. conduktor-poc-kafka-protocol

   6 2 63 6.6 Java

   POC to demonstrate how to alter incoming/outgoing records in Kafka. It's a toy, don't use it in production.

   

   Project mention: The Data Security Duo: Data Encryption and Vulnerability Scans | dev.to | 2024-07-28

   Sidecar/Proxy Approach: Passing sensitive data through a proxy for additional encryption processing is another approach. While this can be effective, deploying sidecars or proxies can be challenging depending on the infrastructure setup. Additionally, data security often needs to be schema-aware, making it difficult for sidecar or proxy layers to handle without additional client-side implementation. Despite these challenges, this approach is framework and client-agnostic, making it easier to implement across diverse data ecosystems. Examples of such offerings include Conduktor.

8. ansilo

   7 1 16 2.4 Rust

   A Postgres interface into any database

   

9. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

10. trino_data_mesh

    8 1 8 1.8

    Proof of concept on how to gain insights with Trino across different databases from a distributed data mesh

11. data-mesh-yelp

    9 1 5 2.0 Smarty

    A complete data product project used to study concepts related to data enginerring such as modeling, collection, operations and so on.

    

    Project mention: You get what you Measure: Understanding your applications health with Grafana, Loki and Prometheus | dev.to | 2024-04-24

    Prometheus can be deployed using the Prometheus Helm Chart. This helm chart contains a lot of features such as the already mentioned Push Gateway, Alert Manager and so on. For simplicity reasons of this tutorial I will not show all the Helm chart configuration but you can see a real example used by me here.

12. data-mesh-demo

    10 1 2 2.0 Java

    A Data Mesh proof-of-concept built on Confluent Cloud

    

data-mesh related posts

• What even is data mesh

  2 projects | news.ycombinator.com | 29 Jul 2021

Index

Sponsored

CodeRabbit: AI Code Reviews for Developers

Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

coderabbit.ai