Detectorist-scraper Alternatives
Similar projects and alternatives to detectorist-scraper
-
ArchiveBox
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
-
grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
forum-dl
Scrape posts, threads from forums, news aggregators, mail archives, export to JSONL, mailbox, WARC
-
warctools
Command line tools and libraries for handling and manipulating WARC files (and HTTP contents) (by internetarchive)
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
detectorist-scraper reviews and mentions
-
Ask HN: How can I back up an old vBulletin forum without admin access?
> I'm just not sure what the intermediate steps would be to get something usable like a vBulletin…
Once you have a crawl, you'll likely want to convert that unstructured data to structured data. For example, if I look at https://www.vbulletin.org/forum/portal.php, the thread title and hierarchy is in
, posts are in, etc. I see an old project (https://github.com/IanLondon/detectorist-scraper) that did this and may be a useful place to start, and I imagine there are others.Once you have the structured data, You can decide whether to use it to build a static site, to import it into another forum, etc.
Stats
The primary programming language of detectorist-scraper is Python.
Sponsored