cryptostore
DataEngineeringProject
| cryptostore | DataEngineeringProject | |
|---|---|---|
| 35 | 5 | |
| 382 | 985 | |
| - | - | |
| 5.4 | 0.0 | |
| 13 days ago | over 1 year ago | |
| Python | Python | |
| GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cryptostore
- cryptostore: NEW Data - star count:350.0
- Anyone tried cryptotick.com ?
-
Data service that streams data into our postgres table
BMoscon to the rescue https://github.com/bmoscon/cryptostore
- cryptostore: NEW Data - star count:316.0
DataEngineeringProject
- What are your favourite GitHub repos that shows how data engineering should be done?
- Is it me or are beginner-friendly ETL pipeline guides that explain from the ground-up how to incorporate the use of various technologies notoriously difficult to find.
-
Starting A Data Engineering Project Series
News RSS Feeds
-
5 Data Sources for Data Engineering Projects
Lastly, the most readily available data source would be data scraped from the internet. To be slightly less vague, I have outlined a project that web-scrapes new online articles every ten minutes to provide all the latest news curated into one place. This project utilizes a wide variety of relevant data engineering tools, which makes it a great project example. The author of this project is Damian Kliś, and he outlines his model architecture below:
-
Can You Recommend Good Data Engineering Projects
Here is my project that got me a few interviews so far: https://github.com/damklis/DataEngineeringProject
What are some alternatives?
cryptofeed - Cryptocurrency Exchange Websocket Data Feed Handler
blinkist-scraper - 📚 Python tool to download book summaries and audio from Blinkist.com, and generate some pretty output
dev-setup - macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
synapse-s3-storage-provider - Synapse storage provider to fetch and store media in Amazon S3
guane-intern-fastapi - FastAPI-PostgreSQL-Celery-RabbitMQ-Redis bakcend with Docker containerization
yaetos - Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
tweets-docker-pipeline - Docker pipeline for streaming tweets and their sentiment score to a Slack channel
amazon-s3-find-and-forget - Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR)
NewsBlur - NewsBlur is a personal news reader that brings people together to talk about the world. A new sound of an old instrument.
Zillow-Data-Engineering
FeedHQ - FeedHQ is a web-based feed reader
openwisp-monitoring - Network monitoring system written in Python and Django, designed to be extensible, programmable, scalable and easy to use by end users: once the system is configured, monitoring checks, alerts and metric collection happens automatically.