snscrape
TweetScraper
Our great sponsors
snscrape | TweetScraper | |
---|---|---|
29 | 4 | |
4,185 | 972 | |
- | - | |
7.3 | 10.0 | |
5 months ago | almost 3 years ago | |
Python | Python | |
GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
snscrape
- API to scrape tweets
-
Twitter scraping for complete profiles (very large data sets)?
Try Snscrape.
- snscrape getting blocked from twitter
- [Project]Topic modelling of tweets from the same user
- Show HN: Twitter API Reverse Engineered
-
Recommend tool other than Maltego ?
If you're looking for small, standalone tools, you can have a look at tools like snscrape (https://github.com/JustAnotherArchivist/snscrape) and WhatsMyName (https://github.com/WebBreacher/WhatsMyName).
- Anyone familiar with the open-source Twitter API alternative Twint?
-
import requests
snscrape is a great program for scraping social media
-
“Starting February 9, we will no longer support free access to the Twitter API”
I've been using this https://github.com/JustAnotherArchivist/snscrape and with 5 threads and no vpn just my laptop and getting about 1 million tweets a day (vs 1000 before being rate limited on the api).
-
Twitter archiver: Make your own simple, public, searchable Twitter archive
I hope you’ll get a better answer, but a minimal solution is to use snscrape (https://github.com/JustAnotherArchivist/snscrape) to download a raw JSON dump of a user timeline. Some caveats: 1. This doesn’t work for all users (e.g. some accounts seem to remain deindexed after unsuspensions). 2. While you can get rid of 90% of the file size by removing fluff columns (e.g. related to processing media and emojis), keep a backup. When importing such JSON, I unwittingly did a lossy data type conversion, and it can be irreparable once the tweets are deleted.
$ snscrape --progress --jsonl twitter-user jack > jack.json
TweetScraper
- Show HN: Twitter API Reverse Engineered
-
No more Twitter notifications from freemium bots: Twitter is discontinuing their free API
try https://github.com/jonbakerfish/TweetScraper or https://github.com/mahrtayyab/tweety for python
-
Is there a tool that lets you download your liked tweets or get them to an excel file?
I'm personally I huge fan of https://github.com/jonbakerfish/TweetScraper on GitHub. It's a Go library that allows you to easily scrape tweets. It unfortunately does require programming knowledge of Go. I've used it for a small "download to json + media" folder program and it worked great for that. Planning on implementing it more on my bigger social media self hosting project I'm working on, but it's a fantastic library if you know programming.
-
Changes to Sourcecatcher after Twitter disables free API access
https://github.com/evanc577/tweet-scraper The technique is largely adapted from TweetScraper but instead of saving to a directory, results are outputted to stdout as json.
What are some alternatives?
facebook_page_scraper - Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
TWINT - An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
instagram_hunter - Instagram-Hunter is a simple tool that helps you find instagram accounts.
reddit-detective - Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
Socialhome - A federated social home
webtoondl - Python webcomics scraper
privalise - Social Media Network Focuses On Data Security And Being Community Driven Web App
linkedin-visualizer - The missing feature in LinkedIn
nider - Python package to add text to images, textures and different backgrounds
LinkScope_Client - Repository for the LinkScope Client software.
tweety - Twitter Scraper
Mastonet - C# Library for Mastodon