How is ArchiveBox?

This page summarizes the projects mentioned and recommended in the original post on reddit.com/r/selfhosted
#Python #Crawler #Web Crawling #Archiving and Digital Preservation (DP) #Pocket

Our great sponsors
  • SonarQube - Static code analysis for 29 languages.
  • Scout APM - Less time debugging, more time building
  • SaaSHub - Software Alternatives and Reviews
Our great sponsors

  • browsertrix-crawler

    Run a high-fidelity browser-based crawler in a single Docker container

    If you need more advanced recursive spider/crawling ability beyond --depth=1, check out Browsertrix, Photon, or Scrapy and pipe the outputted URLs into ArchiveBox.

  • Scrapy

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    If you need more advanced recursive spider/crawling ability beyond --depth=1, check out Browsertrix, Photon, or Scrapy and pipe the outputted URLs into ArchiveBox.

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • Photon

    Incredibly fast crawler designed for OSINT. (by s0md3v)

    If you need more advanced recursive spider/crawling ability beyond --depth=1, check out Browsertrix, Photon, or Scrapy and pipe the outputted URLs into ArchiveBox.

  • ArchiveBox

    🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

    From https://archivebox.io/:

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts