snscrape
nider
Our great sponsors
| snscrape | nider | |
|---|---|---|
| 29 | 1 | |
| 4,215 | 148 | |
| - | - | |
| 7.3 | 0.0 | |
| 5 months ago | 16 days ago | |
| Python | Python | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
snscrape
- API to scrape tweets
-
Twitter scraping for complete profiles (very large data sets)?
Try Snscrape.
- snscrape getting blocked from twitter
- [Project]Topic modelling of tweets from the same user
- Show HN: Twitter API Reverse Engineered
-
Recommend tool other than Maltego ?
If you're looking for small, standalone tools, you can have a look at tools like snscrape (https://github.com/JustAnotherArchivist/snscrape) and WhatsMyName (https://github.com/WebBreacher/WhatsMyName).
- Anyone familiar with the open-source Twitter API alternative Twint?
-
import requests
snscrape is a great program for scraping social media
-
“Starting February 9, we will no longer support free access to the Twitter API”
I've been using this https://github.com/JustAnotherArchivist/snscrape and with 5 threads and no vpn just my laptop and getting about 1 million tweets a day (vs 1000 before being rate limited on the api).
-
Twitter archiver: Make your own simple, public, searchable Twitter archive
I hope you’ll get a better answer, but a minimal solution is to use snscrape (https://github.com/JustAnotherArchivist/snscrape) to download a raw JSON dump of a user timeline. Some caveats: 1. This doesn’t work for all users (e.g. some accounts seem to remain deindexed after unsuspensions). 2. While you can get rid of 90% of the file size by removing fluff columns (e.g. related to processing media and emojis), keep a backup. When importing such JSON, I unwittingly did a lossy data type conversion, and it can be irreparable once the tweets are deleted.
$ snscrape --progress --jsonl twitter-user jack > jack.json
nider
We haven't tracked posts mentioning nider yet.
Tracking mentions began in Dec 2020.
What are some alternatives?
facebook_page_scraper - Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
TWINT - An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
instagram_hunter - Instagram-Hunter is a simple tool that helps you find instagram accounts.
reddit-detective - Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
Socialhome - A federated social home
awesome-slugify - Python flexible slugify function
TextDistance - 📐 Compute distance between sequences. 30+ algorithms, pure python implementation, common interface, optional external libs usage.
webtoondl - Python webcomics scraper
jellyfish - 🪼 a python library for doing approximate and phonetic matching of strings.
privalise - Social Media Network Focuses On Data Security And Being Community Driven Web App
pyparsing - Python library for creating PEG parsers [Moved to: https://github.com/pyparsing/pyparsing]
汉字拼音转换工具（Python 版） - 汉字转拼音(pypinyin)