cryptostore
tweets-docker-pipeline
cryptostore | tweets-docker-pipeline | |
---|---|---|
35 | 2 | |
382 | 2 | |
- | - | |
5.4 | 0.0 | |
13 days ago | over 2 years ago | |
Python | Python | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cryptostore
- cryptostore: NEW Data - star count:350.0
- Anyone tried cryptotick.com ?
-
Data service that streams data into our postgres table
BMoscon to the rescue https://github.com/bmoscon/cryptostore
- cryptostore: NEW Data - star count:316.0
tweets-docker-pipeline
-
Building a dockerized ETL pipeline for streaming positive tweets
First, I created an app on Twitter and got my credentials (API key and Access Token). Then, I wrote the Python code for streaming live tweets, using tweepy with my Twitter credentials. I chose to stream the hashtag #OnThisDay (thought it would be interesting to get a daily notification of what happened years ago) and collected the tweet text and user handle.
-
Automate your data processing pipeline in 9 steps ⚙️
I was really excited, though also a bit overwhelmed by all the things I had to set up for this project. In total, I spent five days learning the tools, debugging, and building this pipeline with Python (including libraries like Tweepy, TextBlob, VADER, and SQLAlchemy), Postgres, MongoDB, Docker, and Airflow (most frustrating part...). If you're interested to see how I did this, you can check out the project on GitHub and read this blog post.
What are some alternatives?
cryptofeed - Cryptocurrency Exchange Websocket Data Feed Handler
twurl - OAuth-enabled curl for the Twitter API
dev-setup - macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
vaderSentiment - VADER Sentiment Analysis. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media, and works well on texts from other domains.
guane-intern-fastapi - FastAPI-PostgreSQL-Celery-RabbitMQ-Redis bakcend with Docker containerization
redditcoins-backend - Pull reddit data from APIs and store it in local db
DataEngineeringProject - Example end to end data engineering project.
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
NewsBlur - NewsBlur is a personal news reader that brings people together to talk about the world. A new sound of an old instrument.
Docker Compose - Define and run multi-container applications with Docker
FeedHQ - FeedHQ is a web-based feed reader
PostgreSQL - Mirror of the official PostgreSQL GIT repository. Note that this is just a *mirror* - we don't work with pull requests on github. To contribute, please see https://wiki.postgresql.org/wiki/Submitting_a_Patch