analytics VS datahub

Take a look at gitlab's dbt project: https://gitlab.com/gitlab-data/analytics/-/blob/master/transform/snowflake-dbt/models/common/schema.yml

A good reference is the Gitlab data team repo. https://gitlab.com/gitlab-data/analytics

For example, their Data Team have a public repository, with a bunch of information on how they organize DAGs, machine learning projects, system configuration, etc.

So I've run through the DBT tutorial stuff and looked over some fairly complex uses of it i.e. GitLab Data and I was wondering if anyone has any opinions or insights into the use of jinja templating in the sql?

Gitlab has their DBT repo open source and is very useful for seeing how to structure a project at scale. https://gitlab.com/gitlab-data/analytics/-/tree/master/transform/snowflake-dbt

Currently the team is working hard on this: https://gitlab.com/gitlab-data/analytics/-/issues/9508

GitLab's dbt project is an excellent example of a mature project at scale. They also have a comprehensive guide to their methodology.

Sounds like you are looking for a data catalog tool instead of db schema management tool. You can check out Amundsen (https://www.amundsen.io/), DataHub (https://datahubproject.io/)

If you are looking for schema change management tool, then you can check out Bytebase (bytebase.com). But it can't answer questions like "which collections contain links to bigmongo.user.id?"

IIUC DataHub (open source project out of LinkedIn) might be relevant here

I’m a huge fan of DataHub, the open source data catalogue spun out of LinkedIn, but it’s best thought of as an observability layer for data assets that can be shared by data engineers and analyst-types. For data users: it’s a stellar search/discovery interface (what datasets are there on this keyword, which are most broadly used across the organization, what downstream products are made with this data, what’s it usually joined to, are it’s upstream pipelines reliable). For data engineers, it’s a comprehensive asset cataloger, crawling your warehouse, orchestrator, modeling layers, features, and reports, matching the lineage into a graph where it can.

If you are looking for a data cataloguing solution, look at Datahub. Haven't used it, but heard good things about it.

What I am looking for is a solution (similar to Amundsen or [Datahub](https://datahubproject.io/)) that also allows to add tables and their metadata manually.

Hello everyone, I'm looking for an open-source data lineage app (e.g. tokern, datahubproject, openmetadata).

What about DataHub? Haven't really used it but I'm actively reading about it and about to use it for some light documentation for some small pipelines.