datahub VS OpenMetadata

There are several commercial providers, but I would definitely recommend Data Hub Project. DataHub Project is an open-source metadata platform that serves as an extensible data catalog and supports data discovery, data observability, and federated governance to address the complexity of the data ecosystem. The data catalog enables the combination of technical, operational and business metadata to provide a 360-degree view of data entities. DataHub makes it possible to pre-enrich important metadata using shift-left practices and respond to changes in real time.

Sounds like you are looking for a data catalog tool instead of db schema management tool. You can check out Amundsen (https://www.amundsen.io/), DataHub (https://datahubproject.io/)

If you are looking for schema change management tool, then you can check out Bytebase (bytebase.com). But it can't answer questions like "which collections contain links to bigmongo.user.id?"

IIUC DataHub (open source project out of LinkedIn) might be relevant here

I’m a huge fan of DataHub, the open source data catalogue spun out of LinkedIn, but it’s best thought of as an observability layer for data assets that can be shared by data engineers and analyst-types. For data users: it’s a stellar search/discovery interface (what datasets are there on this keyword, which are most broadly used across the organization, what downstream products are made with this data, what’s it usually joined to, are it’s upstream pipelines reliable). For data engineers, it’s a comprehensive asset cataloger, crawling your warehouse, orchestrator, modeling layers, features, and reports, matching the lineage into a graph where it can.

If you are looking for a data cataloguing solution, look at Datahub. Haven't used it, but heard good things about it.

What I am looking for is a solution (similar to Amundsen or [Datahub](https://datahubproject.io/)) that also allows to add tables and their metadata manually.

Hello everyone, I'm looking for an open-source data lineage app (e.g. tokern, datahubproject, openmetadata).

What about DataHub? Haven't really used it but I'm actively reading about it and about to use it for some light documentation for some small pipelines.

* It seems like DataHub has an async Kafka ingestion approach while OpenMetadata is API

We do not use Kafka by default. If someone needs kafka they can add it. However for Metadata APIs, we do not feel like Kafka is needed. Lot of projects are getting dependent on Kafka and calling it as real-time. Its unnecessary burden on users who are going to operate in production for 99% of use-cases Kafka is not needed, coming from a Kafka committer :)

2. Yes all of our APIs and Entity definitions are generated using JsonSchema. For us, Json Schema has been awesome, all of our backend / ingestion and UI is generated from JsonSchema and its easy to extend and add new models when needed

3. IMO, we have much more coverage , you can look at the types available here https://github.com/open-metadata/OpenMetadata/tree/main/open... and we are support JsonSchema as a type from a long time

In this blog post, I have demonstrated how I addressed the challenge of dynamically adjusting the height of a textarea element based on its content, preventing the need for vertical scrolling in the title section of the OpenMetadata Knowledge article page.

How does this compare with https://github.com/open-metadata/OpenMetadata

u/cmcau try https://open-metadata.org much easier to setup , for details https://docs.open-metadata.org and for any support https://slack.open-metadata.org

Then, a pool of tools appeared on the market with features that allow covering some of the challenges cited, especially those related to data cataloging. Informatica's tool is perhaps the best known among the licensed. Among the open source tools, I highlight Data Hub (www.datahubproject.io) developed on LinkedIn, Open Metadata (https://open-metadata.org/) and Amundsen (https://www.amundsen.io /) powered by Lyft. In addition to cataloging and discovering data artifacts, these tools allow for a view of data lineage, including technical documentation and business terms, and building relationships between data artifacts. Also, it is possible to register data owners, the people responsible for the data in those tools. This greatly facilitates access request and evaluation process (which today is a major bottleneck).

Probably, in order to have it integrate with tools like OpenLineage and OpenMetadata and such I will have to make open-source contributions.

We are the team behind OpenMetadata . Our mission is to build a centralized metadata platform that offers data discovery, collaboration, governance and quality. We believe that having tool for each of these categories not only result user frustration but metadata silos.

As anyone ever tried open metadata for data QA testing? Curious about that https://open-metadata.org/

We're looking to PoC https://open-metadata.org/ shortly