Twitter archiver: Make your own simple, public, searchable Twitter archive

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • twitter-archiver

    Make your own simple, public, searchable Twitter archive

  • twitter-archive

    Safely host your tweets on your own site (by donohoe)

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • twitvault

    Easily Archive and Search Your Twitter Data with our Syncable Desktop App

  • Another shameless plug: I made a cross platform "native (Rust with Webkit)" desktop app that does the same thing (Win/Lin/Mac). It also allows you to see your "profile" afterwards in the app, search within your Tweets, and allows backing up things like mentions, responses, likes, etc:

    https://github.com/terhechte/twitvault

  • snscrape

    A social networking service scraper in Python

  • I hope you’ll get a better answer, but a minimal solution is to use snscrape (https://github.com/JustAnotherArchivist/snscrape) to download a raw JSON dump of a user timeline. Some caveats: 1. This doesn’t work for all users (e.g. some accounts seem to remain deindexed after unsuspensions). 2. While you can get rid of 90% of the file size by removing fluff columns (e.g. related to processing media and emojis), keep a backup. When importing such JSON, I unwittingly did a lossy data type conversion, and it can be irreparable once the tweets are deleted.

    $ snscrape --progress --jsonl twitter-user jack > jack.json

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts