WebCrawlerForOnlineInflation
practical-data-engineering
| WebCrawlerForOnlineInflation | practical-data-engineering | |
|---|---|---|
| 1 | 4 | |
| 169 | 455 | |
| - | 6.6% | |
| 10.0 | 7.7 | |
| almost 4 years ago | about 2 months ago | |
| Python | Jupyter Notebook | |
| - | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
WebCrawlerForOnlineInflation
-
✨ 5 Open Source Data Engineering Projects 🔥
4️⃣ Web Crawler For Online Inflation
practical-data-engineering
- Show HN: Hands-On Data Engineering with a Real-Estate Project Guide
-
What's your favorite end-to-end tech stack?
Or in a side project: I combined an extensive amount of tools for web-scraping real-estates, uploading them to S3 with MinIO, Spark, and Delta Lake, adding some Data Science magic with Jupyter Notebooks, ingesting into Data Warehouse Apache Druid, visualizing dashboards with Superset and managing everything with Dagster (blog, github)
-
✨ 5 Open Source Data Engineering Projects 🔥
2️⃣ Building a Data Engineering Project in 20 Minutes
-
Building a Data Engineering Project in 20 Minutes
The source-code you can find on practical-data-engineering for the data pipeline or in data-engineering-devops with all it’s details to set things up. Although not all is finished, you can observe the current status of the project on real-estate-project.
What are some alternatives?
HashtagCashtag - My Insight Data Engineering Fellowship project. I implemented a big data processing pipeline based on lambda architecture, that aggregates Twitter and US stock market data for user sentiment analysis using open source tools - Apache Kafka for data ingestions, Apache Spark & Spark Streaming for batch & real-time processing, Apache Cassandra f or storage, Flask, Bootstrap and HighCharts f or frontend.
faros-community-edition - BI, API and Automation layer for your Engineering Operations data
PANDAS-TUTORIAL - Jupyter Notebooks and Data Sets for Pandas Library
ngods-stocks - New Generation Opensource Data Stack Demo
Data-Engineering-Projects - Personal Data Engineering Projects
open-data-stack - Open Data Stack Projects: Examples of End to End Data Engineering Projects
data-engineering-devops - Full stack data engineering tools and infrastructure set-up
go-hnsw
Hugo - The world’s fastest framework for building websites.