metadata-guardian
grai-core
| metadata-guardian | grai-core | |
|---|---|---|
| 1 | 6 | |
| 18 | 270 | |
| - | 1.1% | |
| 7.0 | 9.5 | |
| about 2 months ago | 1 day ago | |
| Python | Python | |
| Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
metadata-guardian
grai-core
-
Launch HN: Grai (YC S22) – Open-Source Data Observability Platform
Elastic v2 if one is interested in such things: https://github.com/grai-io/grai-core/blob/v0.1.33/LICENSE
-
Standalone lineage tool
I’m not sure if this is precisely what you’re looking for but Grai might serve your needs. The backend data model allows you to push any arbitrary metadata you want / need onto the lineage graph and retrieve it either through the rest or graph API. I’m one of the authors so happy to answer any questions you might have.
-
Data Load Diagram
We've been looking at building something like this for Grai specifically to support Airflow but haven't yet prioritized it.
-
Grai, a self-hosted data lineage tool. Test downstream impact of data migration changes
We were frustrated because although we had tests in our data warehouse, they only notified us after an outage occurred. What we needed was a way to detect changes during CI/CD, so we could fix things before they impacted production. So we developed Grai, as an open-source data lineage toolkit pre-built integrations for the most common data stores and designed to work with CI tools, like Github Actions.
What are some alternatives?
pymeta - Utility to download and extract document metadata from an organization. This technique can be used to identify: domains, usernames, software/version numbers and naming conventions.
dbt-snowflake-monitoring - A dbt package from SELECT to help you monitor Snowflake performance and costs
opendatadiscovery-specification - ODD Specification is a universal open standard for collecting metadata.
awesome-data-catalogs - 📙 Awesome Data Catalogs and Observability Platforms.
bragibooks - An audiobook library cleanup and management tool built with Python and Django. Leveraging m4b-merge for audiobook standardization and editing. Ideal for enhancing audiobook library management.
jupysql - Better SQL in Jupyter. 📊
MindsDB - The platform for customizing AI from enterprise data
django-pgschemas - Django multi-tenancy through Postgres schemas
sqlparse - A non-validating SQL parser module for Python
ibis - the portable Python dataframe library
django-pg-upsert - Support Postgres native upsert (INSERT ... ON CONFLICT) for django
Mage - 🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai