How do I find a scraper/crawler that takes a list of URLs and saves those webpages for me?

This page summarizes the projects mentioned and recommended in the original post on /r/DataHoarder

Our great sponsors
  • SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SingleFile

    Web Extension for saving a faithful copy of a complete web page in a single HTML file

  • I don't know if it's exactly what you're looking for, but SingleFile can take a list of URLs to save. It's an extension for Chrome and Firefox that saves an entire webpage to a single, self-contained HTML file. It's easy to use. It can't save dynamic content (like interactive maps) but it can save things like text webpages, articles, blog posts, Old Reddit posts etc. in most cases. It has a "View pending saves" page where you can paste a list of links to save, and it will open and save them 6 at a time.

  • old-reddit-redirect

    Ensure Reddit always loads the old design

  • Pro-Tip: If you want to save Reddit posts I recommend saving them via the old Reddit interface. You can use this extension to always switch to old Reddit. It works with SingleFile while bulk-saving links: https://github.com/tom-james-watson/old-reddit-redirect

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts