dbt-core
monosi
Our great sponsors
dbt-core | monosi | |
---|---|---|
86 | 20 | |
8,718 | 320 | |
6.1% | 1.3% | |
9.7 | 0.0 | |
4 days ago | over 1 year ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dbt-core
-
Relational is more than SQL
dbt integration was one of our major goals early on but we found that the interaction wasn't as straightforward as had hoped.
There is an open PR in the dbt repo: https://github.com/dbt-labs/dbt-core/pull/5982#issuecomment-...
I have some ideas about future directions in this space where I believe PRQL could really shine. I will only be able to write those down in a couple of hours. I think this could be a really exciting direction for the project to grow into if anyone would like to collaborate and contribute!
-
Python: Just Write SQL
I really dislike SQL, but recognize its importance for many organizations. I also understand that SQL is definitely testable, particularly if managed by environments such as DBT (https://github.com/dbt-labs/dbt-core). Those who arrived here with preference to python will note that dbt is largely implemented in python, adds Jinja macros and iterative forms to SQL, and adds code testing capabilities.
-
Transform Your Data Like a Pro With dbt (Data Build Tool)
3). Data Build Tool Repository.
- How do I build a docker image based on a Dockerfile on github?
-
DBT core v1.5 released
Here’s the PR, which includes a what/how/why: https://github.com/dbt-labs/dbt-core/issues/7158
- Building Column Level Lineage for dbt
-
Unit testing with dbt
Hey OP! There are packages like dbt-datamocktool or dbt-unit-testing. You can check it out. You might want to check out this thread as well.
- SQL and M4 = Composable SQL
-
Interview Prep - Senior Data Integration role
RudderStack, dbt, Kafka, Headless CDP, etc. on top of my mind
monosi
-
Open source data observability tools with UI?
I also found https://github.com/monosidev/monosi but it seems there are no activities in the repository from last year.
-
Metadata extraction and management
It’s open source, check out the repository here - https://github.com/monosidev/monosi
-
How to Monitor Supabase with Monosi
🎉 Congratulations, you've just set up and scheduled a data monitor on your Supabase instance. You can now add more monitors to other tables in your database. Find more information on how to use Monosi here.
-
Setting up data monitoring for PostgreSQL
Monosi is an open source data observability and monitoring platform for data teams. It is used to quickly set up monitors on a data store. The monitors run checks for data quality issues and alert on detected anomalies.
Now that you’ve worked through an example using a public PostgreSQL instance, you can further extend this to your own data store. For more information, get started here.
-
Sunday Daily Thread: What's everyone working on this week?
Continuing to build out & stabilize Monosi (open source data observability) - https://github.com/monosidev/monosi
-
Data pipeline suggestions
Observability: Monosi
-
Whats something hot rn or whats going to be next thing we should focus on in data engineering?
Ah ok cool, well I guess you can say a lot of these tools that are becoming big with the modern data stack provide some form of automation. E.g Fivetran / Airbyte extract data on an automated schedule, then you have dbt with the transformations, and then the reverse ETLs like Hightouch / Census that run on an automated a schedule as well. I think it's pretty much becoming somewhat of a standard now to have, e.g. with what I'm building we included a scheduler for automation from the start.
-
Where can I find free data engineering ( big data) projects online?
Ingestion / ETL: Airbyte, Singer, Jitsu Transformation: dbt Orchestration: Airflow, Dagster Testing: GreatExpectations Observability: Monosi Reverse ETL: Grouparoo, Castled Visualization: Lightdash, Superset
-
Is airflow a good pick for monitoring without scheduling?
Got it, yea in terms of monitoring data in snowflake the monosi package can help.
What are some alternatives?
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
metricflow - MetricFlow allows you to define, build, and maintain metrics in code.
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
n8n - Free and source-available fair-code licensed workflow automation tool. Easily automate tasks across different services.
citus - Distributed PostgreSQL as an extension
dagster - An orchestration platform for the development, production, and observation of data assets.
argo-navis - Argo Navis repository for research, docs and misc items
streamlit - Streamlit — A faster way to build and share data apps.
datahub - The Metadata Platform for your Data Stack
targets - Function-oriented Make-like declarative workflows for R
nodejs-bigquery - Node.js client for Google Cloud BigQuery: A fast, economical and fully-managed enterprise data warehouse for large-scale data analytics.
great_expectations - Always know what to expect from your data.