HN-Paywall-Archiver
SingleFile
HN-Paywall-Archiver | SingleFile | |
---|---|---|
1 | 94 | |
5 | 13,721 | |
- | - | |
10.0 | 9.6 | |
almost 2 years ago | 5 days ago | |
JavaScript | JavaScript | |
- | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
HN-Paywall-Archiver
-
Show HN: A userscript that adds archive URLs below the paywalled HN submissions
Source code: https://github.com/MostlyEmre/hn-anti-paywall
Now let me overexplain.
-Why?-
I never liked paywalled articles. I understand where they come from, but I don't like where we cross our paths.
This is why I don't use major news aggregators anymore. Instead, I spend my "catching-up-with-the-world-time" on Hacker News. However, Hacker News (HN) also has its fair-share of paywalled articles. (Around 11.6% according to my short-lived, half-assed attempt at measuring it. See my super old data https://hpa.emre.ca/ I tell the story below.)
-First try-
Around a year ago, when I ran the above experiment, my goal wasn't to run that experiment. It was during my self-teaching & career-changing process, I decided to build a React HN clone. To make it stand-out from the bunch, I added a paywall feature. It would detect paywalled articles and would add an archive URL into the metadata.
The issue with archiving is unless someone archived the link before on the {archiving-project} then the link is most likely not archived. So me sending people to those projects meant nothing. It kinda meant something for me from an ideological standpoint but I assume you are not me.
This rubbed me the wrong way. I decided to build a backend (See https://github.com/MostlyEmre/HN-Paywall-Archiver) that would scan the links and automatically to detect paywalls close to real-time and submit paywalled ones to archive.is for archival. I used Nodejs, Firebase, and React. I was -still am- really proud because I believed it was doing public good in terms of digital preservation. Only 1 person needed to run this script to benefit everyone. As an extra, I was curious on how many paywalled articles were being shared, by whom, at what time. So I also created some analytics functionality to gather the data. And later created a UI to present it.
HN-Paywall-Archiver was great but I stopped running the backend at some point. Because at that point couldn't find a way to continuously run my backend code on some platform for cheap or didn't try hard enough.
P.S. Recently I've been thinking of remaking this version with Cloudflare Workers.
-Hacker News Paywall Archiver Userscript-
After almost a year, I got into userscripts. Super great super awesome concept. People seem to hate javascript unless it is presented as a userscript. So I decided to get my hands dirty to create a simple solution that solves the paywall issue on HN without breaking any hearts.
My solution is not perfect as it had to be simple. But here's the rundown.
Pros:
- Does not beg for attention.
SingleFile
-
How SingleFile Transformed My Obsidian Workflow
That's interesting. I have been saving articles as PDF files, which is browser-independent, but useful just for search and reference, a nuisance to quote/copy-and-paste.
If I search only the computer, I don't get results from EBay and Amazon at the top. The idea of keeping the knowledge base separate from the primary notes is a good idea. In my case, that knowledge base is the file system, and the primary notes are whatever I choose.
When I was using Evernote, the inbox was the knowledge base and notebooks were the focus. I just had too many different potential projects going on to manage this well.
Looking to focus.
I'll revisit Firefox and SingleFile.
Explanation of the zip file inside.
https://github.com/gildas-lormeau/SingleFile/blob/master/faq...
-
Webpage is also a PNG file and a ZIP file
[2] https://github.com/gildas-lormeau/SingleFile/blob/master/faq...
-
My website is one binary
I agree it would be "great" a complete website in the ZIP. I think this is technically possible, someone just have to code it.
[1] https://github.com/gildas-lormeau/SingleFile#singlefile
-
Omnivore – free, open source, read-it-later App
Singlefile [1] works pretty well for me for that use case.
It has the added advantage that the file format is just plain HTML, and together with “reader mode” in most browsers, it’s a great way to save long-form text or other mostly static pages for later reference.
It obviously doesn’t work for very dynamic pages, let alone web apps.
[1] https://github.com/gildas-lormeau/SingleFile
-
Pocket: It gets worse the more you use it
I’ve tried all the third party services for archiving interesting things over the years but nothing beats saving everything to your local filesystem using [SingleFile](https://github.com/gildas-lormeau/SingleFile) and using a full-text search front over the directory (something like Houdahspot, for example).
- 11. 使用浏览器插件保存完整网页
-
How to easily and quickly save all my subbreddit's wikis?
If you want to save them as a file locally you could use something like SingleFile. You could also put the URL for each wiki into archive.org's Save Page Now so that anyone can access it. Either way, without scripting, you'll have to do some manual labor to get the URL for each wiki.
- Save webpages into Obsidian (mobile)
- Wayback: Self-hosted archiving service integrated with Internet Archive
-
Ask HN: Looking for a great tool to archive websites
For small numbers of pages, the SingleFile[0] extension for Firefox (WebExtension) is pretty handy. It's not "archival quality", though, if that's the kind of "archiving" you're doing.
[0] https://github.com/gildas-lormeau/SingleFile
What are some alternatives?
DownloadNet - 💾 DownloadNet - All content you browse online available offline. Search through the full-text of all pages in your browser history. ⭐️ Star to support our work!
leetcode-rating-predictor - Leetcode Rating Predictor built with Node. Browser extension and web interface.
freeRep - Bypass repubblica.it and lastampa.it paywall
ArchiveBox - 🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
Archiver - a streaming interface for archive generation
page-ruler-redux - An awesome page ruler extension for google chrome
bypass-paywalls-chrome - Bypass Paywalls web browser extension for Chrome and Firefox.
monolith - ⬛️ CLI tool for saving complete web pages as a single HTML file
hn-anti-paywall - A userscript for Hacker News that adds archive links under submissions that include paywalled articles.
sidebery - Firefox extension for managing tabs and bookmarks in sidebar.
headless-recorder - Chrome extension that records your browser interactions and generates a Playwright or Puppeteer script.
SnappySnippet - Chrome extension that allows easy extraction of CSS and HTML from selected element.