wget-lua

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication. (by ArchiveTeam)
Review webarchiving warc wget Lua Archiving Crawler crawl Crawling Spider archiveteam wget-lua Zstd Ftp Scraper Scraping crawlers Downloader
Source Code
archiveteam.org
wget-lua Reviews
Suggest alternative
Edit details
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
sponsored

Wget-lua Alternatives

Similar projects and alternatives to wget-lua

ArchiveTeam
wget-lua

  1. ArchiveBox

    275 wget-lua VS ArchiveBox

    🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

  2. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  3. browsertrix-crawler

    13 wget-lua VS browsertrix-crawler

    Run a high-fidelity browser-based web archiving crawler in a single Docker container

  4. os

    11 wget-lua VS os

    Discontinued Tiny Linux distro that runs the entire OS as Docker containers

  5. libarchive

    37 wget-lua VS libarchive

    Multi-format archive and compression library

  6. bitextor

    2 wget-lua VS bitextor

    Bitextor generates translation memories from multilingual websites

  7. fetchurls

    4 wget-lua VS fetchurls

    A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

  8. grab-site

    35 wget-lua VS grab-site

    The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

  9. 7-Zip-zstd

    41 wget-lua VS 7-Zip-zstd

    7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard

  10. pgBackRest

    23 wget-lua VS pgBackRest

    Discontinued Reliable PostgreSQL Backup & Restore

  11. Crawly

    2 wget-lua VS Crawly

    Crawly, a high-level web crawling & scraping framework for Elixir.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better wget-lua alternative or higher similarity.
Suggest an alternative to wget-lua

wget-lua discussion

Log in or Post with

wget-lua reviews and mentions

Posts with mentions or reviews of wget-lua. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-02-10.
  • Alternative to HTTrack (website copier) as of 2023?
    4 projects | /r/DataHoarder | 10 Feb 2023
    You're using it wrong, rtfm, wget is still the standard. It's also extensible beyond the base feature set, take for example wget-lua ArchiveTeams well maintained go to for near all scraping projects by the group.
  • Kiwix - Access Wikipedia (And More) With no Internet
    1 project | /r/selfhosted | 4 Dec 2021
    There are updates changed names but still use more frequent updates than the dumps to get started. I know there is kiwix and xowa. Could probably build it up to current and use wget-at to scrap wikipedia solo. If you want it in html it'll proboy only be a hundred 100TB give or take. I'm wondering if any of the groups are still active on IRC. Saw mentions of a few but I lost my place in all the mobile chrome tabs.

Stats

Basic wget-lua repo stats
2
137
2.5
5 months ago

ArchiveTeam/wget-lua is an open source project licensed under GNU General Public License v3.0 only which is an OSI approved license.

The primary programming language of wget-lua is C.

Popular Comparisons

  1. wget-lua VS browsertrix-crawler
  2. wget-lua VS os
  3. wget-lua VS libarchive
  4. wget-lua VS bitextor
  5. wget-lua VS fetchurls
  6. wget-lua VS grab-site
  7. wget-lua VS ArchiveBox
  8. wget-lua VS 7-Zip-zstd
  9. wget-lua VS pgBackRest
  10. wget-lua VS Crawly

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Loading...