tweets-docker-pipeline
PostgreSQL
tweets-docker-pipeline | PostgreSQL | |
---|---|---|
2 | 411 | |
2 | 14,788 | |
- | 2.3% | |
0.0 | 10.0 | |
over 2 years ago | 6 days ago | |
Python | C | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tweets-docker-pipeline
-
Building a dockerized ETL pipeline for streaming positive tweets
First, I created an app on Twitter and got my credentials (API key and Access Token). Then, I wrote the Python code for streaming live tweets, using tweepy with my Twitter credentials. I chose to stream the hashtag #OnThisDay (thought it would be interesting to get a daily notification of what happened years ago) and collected the tweet text and user handle.
-
Automate your data processing pipeline in 9 steps ⚙️
I was really excited, though also a bit overwhelmed by all the things I had to set up for this project. In total, I spent five days learning the tools, debugging, and building this pipeline with Python (including libraries like Tweepy, TextBlob, VADER, and SQLAlchemy), Postgres, MongoDB, Docker, and Airflow (most frustrating part...). If you're interested to see how I did this, you can check out the project on GitHub and read this blog post.
PostgreSQL
-
Essential Tools & Technologies for New Developers
Every time I buy a new computer, the first thing I install is the servers for MySql and Postgres, the two most common databases. This way, I can start the databases with a simple command like this:
- OpenBSD 7.3 を 7.4 へ アップグレード
-
What do you want to watch next? This is why I built GoodWatch.
Data Handling: Utilizes Windmill for data pipelines, with a primary database powered by PostgreSQL. Auxiliary data storage is handled by MongoDB, with Redis for caching to optimize performance
-
System Design: Databases and DBMS
PostgreSQL
- Presentación del Operador LMS Moodle
-
Introducing LMS Moodle Operator
The LMS Moodle Operator serves as a meta-operator, orchestrating the deployment and management of Moodle instances in Kubernetes. It handles the entire stack required to run Moodle, including components like Postgres, Keydb, NFS-Ganesha, and Moodle itself. Each of these components has its own Kubernetes Operator, ensuring seamless integration and management.
-
Integrate txtai with Postgres
Another key feature of txtai is being able to quickly move from prototyping to production. This article will demonstrate how txtai can integrate with Postgres, a powerful, production-ready and open source object-relational database system. After txtai persists content to Postgres, we'll show it can be directly queried with SQL from any Postgres client
-
Understanding SQL vs. NoSQL Databases: A Beginner's Guide
SQL (Structured Query Language) databases are relational databases. They organize data into tables with rows and columns, and they use SQL for querying and managing data. Examples include MySQL, PostgreSQL, and SQLite.
-
From zero to hero: using SQL databases in Node.js made easy
Node.js, MySQL and PostgreSQL servers installed on your machine
-
I Deployed My Own Cute Lil’ Private Internet (a.k.a. VPC)
Each app’s front end is built with Qwik and uses Tailwind for styling. The server-side is powered by Qwik City (Qwik’s official meta-framework) and runs on Node.js hosted on a shared Linode VPS. The apps also use PM2 for process management and Caddy as a reverse proxy and SSL provisioner. The data is stored in a PostgreSQL database that also runs on a shared Linode VPS. The apps interact with the database using Drizzle, an Object-Relational Mapper (ORM) for JavaScript. The entire infrastructure for both apps is managed with Terraform using the Terraform Linode provider, which was new to me, but made provisioning and destroying infrastructure really fast and easy (once I learned how it all worked).
What are some alternatives?
twurl - OAuth-enabled curl for the Twitter API
psycopg2 - PostgreSQL database adapter for the Python programming language
vaderSentiment - VADER Sentiment Analysis. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media, and works well on texts from other domains.
ClickHouse - ClickHouse® is a free analytics DBMS for big data
redditcoins-backend - Pull reddit data from APIs and store it in local db
phpMyAdmin - A web interface for MySQL and MariaDB
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Firebird - FB/Java plugin for Firebird
Docker Compose - Define and run multi-container applications with Docker
Adminer - Database management in a single PHP file
cryptostore - A scalable storage service for cryptocurrency data
SQLAlchemy - The Database Toolkit for Python