Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work. Learn more →
Top 7 Python data-pipeline Projects
-
-
Mage
🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai
Project mention: A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes | dev.to | 2023-06-12In the coming years, Mage will create a cooperative experience so that developers can build data pipelines with their team and level up together. After that journey, Mage will go on an epic quest to create the 1st open world community experience in the data universe.
-
Mergify
Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
-
meltano
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Project mention: meltano VS cloudquery - a user suggested alternative | libhunt.com/r/meltano | 2023-06-02 -
Here's the project: https://github.com/vmware/versatile-data-kit
-
-
-
SmartPipeline
A framework for rapid development of robust data pipelines following a simple design pattern
Project mention: Show HN: SmartPipeline, robust and light data pipelines in Python | news.ycombinator.com | 2023-05-03 -
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
Python data-pipelines related posts
- Experience with Dagster.io?
- Dagster tutorials
- The Dagster Master Plan
- The Why and How of Dagster User Code Deployment Automation
- Mage Battlegrounds: Craft insights from real-time customer behavior analysis
- Looking for an open-source project
- Best Orchestration Tool to run dbt projects?
-
A note from our sponsor - Sonar
www.sonarsource.com | 22 Sep 2023
Index
What are some of the best open-source data-pipeline projects in Python? This list will help you:
Project | Stars | |
---|---|---|
1 | dagster | 8,500 |
2 | Mage | 5,516 |
3 | meltano | 1,185 |
4 | versatile-data-kit | 358 |
5 | dbt-data-reliability | 243 |
6 | patterns-devkit | 106 |
7 | SmartPipeline | 13 |