Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work. Learn more →
Top 7 Python data-pipeline Projects
An orchestration platform for the development, production, and observation of data assets.Project mention: Experience with Dagster.io? | news.ycombinator.com | 2023-07-25
🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-aiProject mention: A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes | dev.to | 2023-06-12
In the coming years, Mage will create a cooperative experience so that developers can build data pipelines with their team and level up together. After that journey, Mage will go on an epic quest to create the 1st open world community experience in the data universe.
Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.Project mention: meltano VS cloudquery - a user suggested alternative | libhunt.com/r/meltano | 2023-06-02
One framework to develop, deploy and operate data workflows with Python and SQL.Project mention: Looking for a data blogger | /r/opensource | 2023-05-19
Here's the project: https://github.com/vmware/versatile-data-kit
Data anomalies monitoring as dbt tests and dbt artifacts uploader.
Data pipelines from re-usable components
A framework for rapid development of robust data pipelines following a simple design patternProject mention: Show HN: SmartPipeline, robust and light data pipelines in Python | news.ycombinator.com | 2023-05-03
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
Python data-pipelines related posts
Experience with Dagster.io?
1 project | news.ycombinator.com | 25 Jul 2023
1 project | /r/dataengineering | 26 Jun 2023
The Dagster Master Plan
2 projects | /r/dataengineering | 16 Jun 2023
The Why and How of Dagster User Code Deployment Automation
1 project | dev.to | 1 May 2023
Mage Battlegrounds: Craft insights from real-time customer behavior analysis
2 projects | dev.to | 10 Apr 2023
Looking for an open-source project
2 projects | /r/dataengineering | 13 Feb 2023
Best Orchestration Tool to run dbt projects?
3 projects | /r/dataengineering | 8 Feb 2023
A note from our sponsor - Sonar
www.sonarsource.com | 22 Sep 2023
What are some of the best open-source data-pipeline projects in Python? This list will help you: