snscrape
jekyll-twitter-plugin
Our great sponsors
| snscrape | jekyll-twitter-plugin | |
|---|---|---|
| 29 | 2 | |
| 4,215 | 238 | |
| - | - | |
| 7.3 | 0.0 | |
| 5 months ago | about 3 years ago | |
| Python | Ruby | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
snscrape
- API to scrape tweets
-
Twitter scraping for complete profiles (very large data sets)?
Try Snscrape.
- snscrape getting blocked from twitter
- [Project]Topic modelling of tweets from the same user
- Show HN: Twitter API Reverse Engineered
-
Recommend tool other than Maltego ?
If you're looking for small, standalone tools, you can have a look at tools like snscrape (https://github.com/JustAnotherArchivist/snscrape) and WhatsMyName (https://github.com/WebBreacher/WhatsMyName).
- Anyone familiar with the open-source Twitter API alternative Twint?
-
import requests
snscrape is a great program for scraping social media
-
“Starting February 9, we will no longer support free access to the Twitter API”
I've been using this https://github.com/JustAnotherArchivist/snscrape and with 5 threads and no vpn just my laptop and getting about 1 million tweets a day (vs 1000 before being rate limited on the api).
-
Twitter archiver: Make your own simple, public, searchable Twitter archive
I hope you’ll get a better answer, but a minimal solution is to use snscrape (https://github.com/JustAnotherArchivist/snscrape) to download a raw JSON dump of a user timeline. Some caveats: 1. This doesn’t work for all users (e.g. some accounts seem to remain deindexed after unsuspensions). 2. While you can get rid of 90% of the file size by removing fluff columns (e.g. related to processing media and emojis), keep a backup. When importing such JSON, I unwittingly did a lossy data type conversion, and it can be irreparable once the tweets are deleted.
$ snscrape --progress --jsonl twitter-user jack > jack.json
jekyll-twitter-plugin
-
“Starting February 9, we will no longer support free access to the Twitter API”
Wow this is disruptive given it's less than 1 week's notice.
I've been using a Jekyll plugin for like 4 years to embed tweets on my site. Nothing crazy, just using: https://github.com/rob-murray/jekyll-twitter-plugin
Since it uses the API I guess that's going to go away. I guess I'll never link another tweet on my site again.
-
Building Jekyll-Twitch, the gem
Jekyll-YouTube Jekyll-Twitter Jekyll-Gist (This one is maintained by Jekyll)
What are some alternatives?
facebook_page_scraper - Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
TWINT - An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
instagram_hunter - Instagram-Hunter is a simple tool that helps you find instagram accounts.
reddit-detective - Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
Socialhome - A federated social home
webtoondl - Python webcomics scraper
privalise - Social Media Network Focuses On Data Security And Being Community Driven Web App
linkedin-visualizer - The missing feature in LinkedIn
nider - Python package to add text to images, textures and different backgrounds
LinkScope_Client - Repository for the LinkScope Client software.
tweety - Twitter Scraper
Mastonet - C# Library for Mastodon