Top 9 Python prefect Projects
-
premier-league
A Data Engineering project. Repository for backend infrastructure and Streamlit app files for a Premier League Dashboard.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
prefect-deployment-patterns
Code examples showing flow deployment to various types of infrastructure
-
magic-the-gathering
A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.
-
youtube_data_analysis
Created an optimised pipeline to provide accurate data for analysis, then used snowsight (provided by Snowflake) to create a dashboard.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
weather_data_pipeline
This is a PySpark-based data pipeline that fetches weather data for a few cities, performs some basic processing and transformation on the data, and then writes the processed data to a Google Cloud Storage bucket and a BigQuery table.The data is then viewed in a looker dashboard
Project mention: Prefect: A workflow orchestration tool for data pipelines | news.ycombinator.com | 2024-03-13
I have a data engineering project that uses BigQuery, Cloud Run, Compute Engine, Cloud SQL, Artifact Registry, Firestore, and Datastream.
Project mention: Created my first Data Engineering Project which integrates F1 data using Prefect, Terraform, dbt, BigQuery and Looker Studio | /r/dataengineering | 2023-07-01Github
Project mention: Building a Weather Data Pipeline with PySpark, Prefect, and Google Cloud | dev.to | 2023-05-01We'll be using PySpark for distributed data processing, Prefect for workflow management, and Google Cloud Storage and BigQuery for data storage and processing.The code is available on github.
Project mention: Ask HN: Open-Source Canary Release Tool – Seeking Your Feedback | news.ycombinator.com | 2023-07-14I'm excited to introduce CanaryPy, a new open-source tool to make your data pipelines more robust by introducing new releases of your data pipelines minimising the impact of unanticipated issues. It has a plugin for Apache Airflow for now but more to come.
We'd love for you to check it out on GitHub: https://github.com/thcidale0808/canarypy
Your feedback and suggestions for improvement are precious to us. What features would you like to see? How's the usability? Would you have any thoughts on integration with your current tools?
Thank you in advance for your insights!
Index
What are some of the best open-source prefect projects in Python? This list will help you:
Project | Stars | |
---|---|---|
1 | Prefect | 14,586 |
2 | premier-league | 142 |
3 | prefect-deployment-patterns | 93 |
4 | magic-the-gathering | 36 |
5 | maternal-health-risk | 35 |
6 | f1-data-pipeline | 23 |
7 | youtube_data_analysis | 16 |
8 | weather_data_pipeline | 3 |
9 | canarypy | 3 |
Sponsored