The Internet Archive is under a DDoS attack

1. bookcorpus

   1 4 836 3.1 Python

   Crawl BookCorpus

   

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. SingleFile

   2 108 18,666 9.2 JavaScript

   Web Extension for saving a faithful copy of a complete web page in a single HTML file

   

   > But I don't think they're around anymore and have no idea how you could achieve similar functionality with dynamic pages anyway.

   Chromium's MHTML "Save as…" and the SingleFile WebExtension should both save copies of the rendered DOM.

   Apparently Safari has WebArchive and Mozilla had MAFF for similar use cases.

   I think WARC is supposed to save enough data about network streams for dynamic pages to work. At least on the Wayback Machine, infinite scrolling and "Load More" buttons do kinda work sometimes. You may have to load the archived pages in a browser and try to use each dynamic feature at least once, to trigger requests for needed resources.

   SingleFile: https://github.com/gildas-lormeau/SingleFile

   LWN on WARC, tools: https://anarc.at/blog/2018-10-04-archiving-web-sites/

   Self-hostable web archives: https://awesome-selfhosted.net/tags/archiving-and-digital-pr...

   Wayback Machine addons, bookmarklets: https://help.archive.org/help/save-pages-in-the-wayback-mach...

4. ArchiveBox

   3 269 24,178 9.8 Python

   🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

   

Suggest a related project