dbt-core
airbyte
Our great sponsors
dbt-core | airbyte | |
---|---|---|
86 | 139 | |
8,718 | 13,646 | |
6.1% | 5.2% | |
9.7 | 10.0 | |
3 days ago | 4 days ago | |
Python | Python | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dbt-core
-
Relational is more than SQL
dbt integration was one of our major goals early on but we found that the interaction wasn't as straightforward as had hoped.
There is an open PR in the dbt repo: https://github.com/dbt-labs/dbt-core/pull/5982#issuecomment-...
I have some ideas about future directions in this space where I believe PRQL could really shine. I will only be able to write those down in a couple of hours. I think this could be a really exciting direction for the project to grow into if anyone would like to collaborate and contribute!
-
Python: Just Write SQL
I really dislike SQL, but recognize its importance for many organizations. I also understand that SQL is definitely testable, particularly if managed by environments such as DBT (https://github.com/dbt-labs/dbt-core). Those who arrived here with preference to python will note that dbt is largely implemented in python, adds Jinja macros and iterative forms to SQL, and adds code testing capabilities.
-
Transform Your Data Like a Pro With dbt (Data Build Tool)
3). Data Build Tool Repository.
- How do I build a docker image based on a Dockerfile on github?
-
DBT core v1.5 released
Here’s the PR, which includes a what/how/why: https://github.com/dbt-labs/dbt-core/issues/7158
- Building Column Level Lineage for dbt
-
Unit testing with dbt
Hey OP! There are packages like dbt-datamocktool or dbt-unit-testing. You can check it out. You might want to check out this thread as well.
- SQL and M4 = Composable SQL
-
Interview Prep - Senior Data Integration role
RudderStack, dbt, Kafka, Headless CDP, etc. on top of my mind
airbyte
-
Who's hiring developer advocates? (October 2023)
Link to GitHub -->
- All the ways to capture changes in Postgres
-
Is it impossible to contribute to open source as a data engineer?
You can try and contribute some new connectors/operators for workflow managers like Airflow or Airbyte
-
airbyte VS cloudquery - a user suggested alternative
2 projects | 2 Jun 20232 projects | 2 Jun 2023
-
New age ETL products every data team needs to know
- https://airbyte.com/
2. Reverse ETL:
-
Is it safe to update docker/docker-compose?
Here's the docker-compose file https://github.com/airbytehq/airbyte/blob/master/docker-compose.yaml
I'm trying to insall https://airbyte.com/ is a great selfhosted ELT platform. In common words, it's an app that can access all kinds of api to scrub the data and put it in a database. I really like the idea of being able to own my data and make all kinds of analyse with it.
-
Top 10 Best Open Source GitHub repos for Developers 2023
AirByte GitHub: https://github.com/airbytehq/airbyte
What are some alternatives?
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dagster - An orchestration platform for the development, production, and observation of data assets.
Prefect - The easiest way to build, run, and monitor data pipelines at scale.
meltano
jitsu - Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
spark-rapids - Spark RAPIDS plugin - accelerate Apache Spark with GPUs
supabase - The open source Firebase alternative.
dbt - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications. [Moved to: https://github.com/dbt-labs/dbt-core]
n8n-docs - Documentation for n8n, a fair-code licensed automation tool with a free community edition and powerful enterprise options. Build AI functionality into your workflows.
superset - Apache Superset is a Data Visualization and Data Exploration Platform
incubator-seatunnel - SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time). [Moved to: https://github.com/apache/seatunnel]
metricflow - MetricFlow allows you to define, build, and maintain metrics in code.