tweets-docker-pipeline
redditcoins-backend
tweets-docker-pipeline | redditcoins-backend | |
---|---|---|
2 | 2 | |
2 | 12 | |
- | - | |
0.0 | 6.4 | |
over 2 years ago | about 2 months ago | |
Python | Python | |
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tweets-docker-pipeline
-
Building a dockerized ETL pipeline for streaming positive tweets
First, I created an app on Twitter and got my credentials (API key and Access Token). Then, I wrote the Python code for streaming live tweets, using tweepy with my Twitter credentials. I chose to stream the hashtag #OnThisDay (thought it would be interesting to get a daily notification of what happened years ago) and collected the tweet text and user handle.
-
Automate your data processing pipeline in 9 steps ⚙️
I was really excited, though also a bit overwhelmed by all the things I had to set up for this project. In total, I spent five days learning the tools, debugging, and building this pipeline with Python (including libraries like Tweepy, TextBlob, VADER, and SQLAlchemy), Postgres, MongoDB, Docker, and Airflow (most frustrating part...). If you're interested to see how I did this, you can check out the project on GitHub and read this blog post.
redditcoins-backend
-
Daily Sentiments on r/CC and Top 10 Picks for the day!
I've added many coins on user requests here on reddit. So will do the same with GET after looking into it. Usually takes a week or two until I get my shit together and do that. If you got any more requests, please comment here / open an issue. For now, adding GET to the list.
What are some alternatives?
twurl - OAuth-enabled curl for the Twitter API
Automatic-Youtube-Reddit-Text-To-Speech-Video-Generator-and-Uploader - A series of 3 programs that will automatically receive scripts from Reddit, allow the user to edit them, then be sent off to a video generator where they will be uploaded to YouTube automatically.
vaderSentiment - VADER Sentiment Analysis. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media, and works well on texts from other domains.
URS - Universal Reddit Scraper - A comprehensive Reddit scraping/archival command-line tool.
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
django-timescaledb - A Django database backend and tooling for Timescaledb.
Docker Compose - Define and run multi-container applications with Docker
dev-setup - macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
PostgreSQL - Mirror of the official PostgreSQL GIT repository. Note that this is just a *mirror* - we don't work with pull requests on github. To contribute, please see https://wiki.postgresql.org/wiki/Submitting_a_Patch
redditsfinder - Archive a reddit user's post history. Formatted overview of a profile, JSON containing every post, and picture downloads. Uses the pushshift API.
cryptostore - A scalable storage service for cryptocurrency data
cheatsheets - My Cheatsheet Repository