dbt-metabase
ngods-stocks
Our great sponsors
| dbt-metabase | ngods-stocks | |
|---|---|---|
| 1 | 3 | |
| 425 | 354 | |
| - | - | |
| 8.2 | 0.0 | |
| 11 days ago | about 1 year ago | |
| Python | Jupyter Notebook | |
| MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dbt-metabase
-
A modern data stack for startups
So how do we get this into Metabase? There's a tool called dbt-metabase that can infer Metabase semantic type information from the dbt schema and push it into Metabase- we run this whenever complete a dbt build, helping sync Metabase with whatever new fields we may have added.
ngods-stocks
-
I'm way over my head
I've worked for 3-4 years in positions where I helped structure ETLs, DWs and alike. However, I'm now on the cusp of being hired to help structure the area in a big investment fund here, helping the research area have an easier time focusing on their models. My previous experience led me to grasp DBT, SQL, and most of my experience came from using a Microsoft stack with SSIS, Analysis Services and the like. I'm feeling wayyyy over my head to start building this, and the multitude of possible stacks make me very afraid that I might overengineer this, and I will initially be alone in the area. What do I do? Fake it till I make it? I never lied in my resume, so it's not like they expect a senior with plenty of experience but still... I read this: https://github.com/zsvoboda/ngods-stocks And it seems like a good starter, albeit overly complex for our use case. I could use suggestions, people to talk to, etc. Please help
-
Apache Iceberg-based opensource analytics stack demo
Hi, I've created an opensource demo of a Docker-based local analytics stack that includes Apache Iceberg, Trino, Spark, Dagster (orchestration), Cube.dev (analytics model), Metabase (reports and dashboards), and Jupyter (data science notebook). I think that this is a pretty good starting point for Iceberg projects. Feel free to check it out at GitHub.
-
Iceberg + Spark + Trino + Dagster: modern, open-source data stack installation
I’m guessing that you use the Spark JDBC dataframes. Trino is in my opinion easier to use. You get SQL access to all pgsql tables with this simple config file. No need to write a piece of code for each table. The config above just maps the pgsql schema to a Trino schema. Then you configure Iceberg with another config file and you can do cross-schema SQL queries like create table pgsql.xyz from select * from iceberg.abc. Or you can use dbt that is based on SQL.
What are some alternatives?
dbt-fal - do more with dbt. dbt-fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.
practical-data-engineering - Practical Data Engineering: A Hands-On Real-Estate Project Guide
dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
amazon-emr-with-delta-lake - Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR
airflow-dbt - Apache Airflow integration for dbt
synapse-azure-data-explorer-101 - Getting started with Azure Synapse and Azure Data Explorer
nodejs-bigquery - Node.js client for Google Cloud BigQuery: A fast, economical and fully-managed enterprise data warehouse for large-scale data analytics.
udacity_bike_share_datalake_project - Azure Data Lake
pgsink - Logically replicate data out of Postgres into sinks (files, Google BigQuery, etc)
data-engineering-zoomcamp - Free Data Engineering course!
MetabaseMonitoringToolkit - Set of queries designed to measure how users are consuming queries and dashboards.
H2O - H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.