data-engineering-nd
Data-Engineering-Projects
| data-engineering-nd | Data-Engineering-Projects | |
|---|---|---|
| 7 | 2 | |
| 8 | 722 | |
| - | - | |
| 0.0 | 10.0 | |
| about 2 years ago | about 1 year ago | |
| Jupyter Notebook | Jupyter Notebook | |
| - | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
data-engineering-nd
-
Data Pipelines explained with Airflow
In the following lines I am doing a write-up about everything I learned about data pipelines at the Udacity online class. It gives a general overview about data pipelines and provides also the core concepts of Airflow and some links to code examples on github.
-
Run Spark locally with Docker
You can find the code also here.
-
Spark for beginners - and you
Coding examples here.
-
Cloud computing quickstart
IaC - Infrastructure as Code Example with boto3
-
Relational data models
If you need a higher resolution please use this page
-
Get started with data engineering
In addition you can find the according exercises on my github account.
-
Structured Query Language
If the resolution here is too low - in case you really want to read it - you can find a higher resolution here.
Data-Engineering-Projects
- Pitanje za data engineering?
-
✨ 5 Open Source Data Engineering Projects 🔥
5️⃣ Data Engineering Projects
What are some alternatives?
udacimak - Udacity Nanodegree and Course Downloader
Udacity-Data-Engineering-Projects - Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
quilt - Quilt is a data mesh for connecting people with actionable data
practical-data-engineering - Practical Data Engineering: A Hands-On Real-Estate Project Guide
migrate - Database migrations. CLI and Golang library.
HashtagCashtag - My Insight Data Engineering Fellowship project. I implemented a big data processing pipeline based on lambda architecture, that aggregates Twitter and US stock market data for user sentiment analysis using open source tools - Apache Kafka for data ingestions, Apache Spark & Spark Streaming for batch & real-time processing, Apache Cassandra f or storage, Flask, Bootstrap and HighCharts f or frontend.
Apache Hadoop - Apache Hadoop
Data-Engineering-Roadmap - Roadmap for Data Engineering
PANDAS-TUTORIAL - Jupyter Notebooks and Data Sets for Pandas Library
data-engineer-roadmap - Roadmap to becoming a data engineer in 2021
WebCrawlerForOnlineInflation - Price Crawler - Tracking Price Inflation