forum-dl
Scrape posts, threads from forums, news aggregators, mail archives, export to JSONL, mailbox, WARC (by mikwielgus)
detectorist-scraper
A scrapy spider to extract post, thread, and user information from a vBulletin forum to a MongoDB database. (by IanLondon)
| forum-dl | detectorist-scraper | |
|---|---|---|
| 4 | 1 | |
| 61 | 30 | |
| - | - | |
| 9.0 | 10.0 | |
| 8 months ago | over 8 years ago | |
| Python | Python | |
| MIT License | - |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
forum-dl
Posts with mentions or reviews of forum-dl.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-01-29.
-
Ask HN: How can I back up an old vBulletin forum without admin access?
You can try forum-dl, a forum scraping tool I've been writing for this purpose: https://github.com/mikwielgus/forum-dl
It's single-threaded, alpha-quality software, and still isn't compatible with many forums and themes. But it can export WARCs and may just happen to work for you.
- Forum-dl – convert forums, news aggregators, mail archives to JSONL/mailbox/WARC
- forum-dl - Download/archive posts and threads from internet forums from command-line
- forum-dl – Download/archive posts/threads from internet forums on command-line
detectorist-scraper
Posts with mentions or reviews of detectorist-scraper.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-01-29.
-
Ask HN: How can I back up an old vBulletin forum without admin access?
> I'm just not sure what the intermediate steps would be to get something usable like a vBulletin…
Once you have a crawl, you'll likely want to convert that unstructured data to structured data. For example, if I look at https://www.vbulletin.org/forum/portal.php, the thread title and hierarchy is in
, posts are in, etc. I see an old project (https://github.com/IanLondon/detectorist-scraper) that did this and may be a useful place to start, and I imagine there are others.Once you have the structured data, You can decide whether to use it to build a static site, to import it into another forum, etc.
What are some alternatives?
When comparing forum-dl and detectorist-scraper you can also consider the following projects:
vbulletin - Ruby Gem to fetch data from vBulletin Forums