Reddit-API-Pipeline
finnhub-streaming-data-pipeline
Reddit-API-Pipeline | finnhub-streaming-data-pipeline | |
---|---|---|
7 | 2 | |
271 | 248 | |
- | - | |
0.0 | 5.6 | |
over 1 year ago | 5 months ago | |
Python | HCL | |
- | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Reddit-API-Pipeline
-
Reddit ELT Pipeline
P.S. I followed steps from this repository but made some adjustments: ABZ-Aaron/Reddit-API-Pipeline (github.com)
- Where can I find online projects end-to-end?
- I created a pipeline extracting Reddit data using Airflow, Docker, Terraform, S3, dbt, Redshift, and Google Data Studio
-
Do you think this AWS based personal project would be suitable and complex enough for a resume?
I recently created one using Airflow, Docker, dbt, S3, Redshift, and PowerBI. It's not perfect, and totally overkill with regards to the tools I used, but you can find it here: https://github.com/ABZ-Aaron/Reddit-API-Pipeline
- Is it worth me applying to Data Engineering roles with this Resume/CV?
finnhub-streaming-data-pipeline
-
Reddit Sentiment Analysis Real-Time* Data Pipeline
I didn't use any specific guide. It was mostly build, test, integrate and repeat for each component. For some of them, I went through official documentation on getting started with each application and implemented it in the cluster. However, I reckon you can find other tutorials to setup each application by itself. A few github projects helped me in planning the project architecture and codebase structure like https://github.com/RSKriegs/finnhub-streaming-data-pipeline and https://gitlab.fit.cvut.cz/kozlovit/ni-dip-project-kozlovit.
- Where can I find online projects end-to-end?
What are some alternatives?
data-engineering-zoomcamp - Free Data Engineering course!
streamify - A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
terraform-cdk - Define infrastructure resources using programming constructs and provision them using HashiCorp Terraform
surf_dash
versatile-data-kit - One framework to develop, deploy and operate data workflows with Python and SQL.
reddit-streaming-pipeline - A real-time reddit data streaming pipeline for sentiment analysis of various subreddits
audiophile-e2e-pipeline - Pipeline that extracts data from Crinacle's Headphone and InEarMonitor databases and finalizes data for a Metabase Dashboard.
udacity-capstone
aws-cdk - The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code