Top 18 wget Open-Source Projects

• ArchiveBox

  248 19,861 9.8 Python

  🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

  

Project mention: Ask HN: What Underrated Open Source Project Deserves More Recognition? | news.ycombinator.com | 2024-03-07

Two projects I greatly appreciate, allowing me to easily archive my bandcamp and GOG purchases (after the initial setup anyways):

https://github.com/easlice/bandcamp-downloader

https://github.com/Kalanyr/gogrepoc

And I recently learned about archivebox, which I think is going to be a fast favorite and finally let me clear out my mess of tabs/bookmarks: https://github.com/ArchiveBox/ArchiveBox

• gdown

  8 3,917 8.4 Python

  Google Drive Public File Downloader when Curl/Wget Fails

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• xidel

  18 652 5.6 Pascal

  Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Project mention: Move over jq I found something easier: fx | news.ycombinator.com | 2023-06-06

You could try Xidel[1]. It supports JSON, XML and HTML using XPath/XQuery 3.1

It has some extensions to the standard that are pretty nice (JSONiq, CSS selectors, html “template” matching), but you can limit it to just standard XPath/XQuery if you like.

I recommend getting the nightly v .99 build if you give it a try, the stable .98 version is pretty old and I’ve had no issues with .99

1. https://www.videlibri.de/xidel.html

• bitextor

  2 279 5.9 Python

  Bitextor generates translation memories from multilingual websites

  

• google-group-crawler

  1 214 0.0 Shell

  [Deprecated] Get (almost) original messages from google group archives. Your data is yours.

• webpages-to-ebook

  4 187 0.0 JavaScript

  Create an EPUB from a list of URLs. Standing on the shoulders of Wget, Readability and Pandoc.

• CuTE

  2 145 9.2 Rust

  HTTP client/libcurl TUI front end in Rust, with request + key storage (by PThorpe92)

Project mention: Show HN: Libcurl TUI front-end in Rust. Developed from my Prison cell | news.ycombinator.com | 2023-11-08

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• fetchurls

  4 123 0.0 Shell

  A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

• pillager

  1 114 0.0 Shell

  A wget script for pillaging.

• aim

  2 92 9.0 Rust

  🎯 A command line download/upload tool with resume. (by mihaigalos)

Project mention: How to copy a file between devices? | news.ycombinator.com | 2024-02-14

If you're fine with plain http on your local network, you can use https://github.com/mihaigalos/aim

Spoiler: I wrote it.

• wget-lua

  2 82 6.1 C

  Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

  

• pynps

  2 61 0.0 Python

  A cli Linux and Windows Nopaystation client made with python and wget. Works with PSvita, PSP, PSMobile and PS3 packages

• Static-Binaries

  1 58 9.2 Tcl

  Cross Compiled Statically Linked Binaries for Android Linux macOS Windows & More (by Azathothas)

Project mention: An independent package manager that every hacker deserves | news.ycombinator.com | 2023-11-29

• pihole-regex

  2 55 9.7 TSQL

  Custom RegEx, Exact, and Adlist filters for Pi-hole's FTLDNS (by slyfox1186)

• kanrisuru

  3 17 0.0 Ruby

  Kanrisuru helps you manage your remote servers with objected oriented ruby. Results come back as structured data, parsed, prepared and ready for you to easily use in your applications.

  

• cl-wget

  1 9 0.0 Common Lisp

  The Non-Interactive Network Downloader: cl-wget is a free software for retrieving files using HTTPS; cl-wget makes mirroring websites easy.

  

• sharc

  1 4 0.0 JavaScript

  A very minimal, unbloated archiving and bookmarking tool (by sadirde)

• static-builds

  1 2 3.7 Shell

  Statically-built, dependency free binaries of software packages for Linux.

Project mention: Show HN: Portable, dependency-free binaries of software packages for Linux | news.ycombinator.com | 2024-04-09

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

wget related posts

• Vice website is shutting down

  1 project | news.ycombinator.com | 23 Feb 2024

• ArchiveBox – open-source self-hosted web archiving

  2 projects | news.ycombinator.com | 13 Jan 2024

• BetaWiki – An open encyclopedia of software history

  1 project | news.ycombinator.com | 20 Jun 2023

• How to Read and Organize Online Articles (Without Driving Yourself Crazy)

  3 projects | news.ycombinator.com | 18 Jun 2023

• Need a way of saving websites

  1 project | /r/DataHoarder | 18 May 2023

• Archivebox - trying to hoard 112K bookmarks

  1 project | /r/selfhosted | 30 Apr 2023

• Is there a way to save every page, every text, every image, absolutely ever single piece of information that comes into my browser to disk. In an orderly, time-stamped, easily mass searchable manner ?

  1 project | /r/firefox | 20 Apr 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 6 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com