ftr-site-config VS arc90-readability

As far as full-text caching... maybe a self-hosted instance or paid version of the FiveFilters Full-Text RSS service would work. You can integrate that into whatever aggregator you want.

However you can retrofit this onto any reader by using a service that creates a full text feed from a summary feed. Two that I have used in the past are https://morss.it/ and https://www.fivefilters.org/full-text-rss/.

RSS feeds that don't contain the full article text drive me nuts.

Here is a workaround that I've had good luck with:

https://www.fivefilters.org/full-text-rss/

In addition to improving usability, it defeats attempts to measure clickbait summary efficacy, etc., since it breaks sites' ability to pull popularity / telemetry info.

By any chance, could this be used as an alternative to the full-article RSS tool that FiveFilters offers?

Please check out FullTextRSS from Five Filters: https://www.fivefilters.org/full-text-rss/

They have an OSS version you can host yourself. It fixes the problem of sites not sharing their full text in their feed, by going and scraping the site into a full feed for you.

Full-Text RSS - FiveFilters.org

This is great!

If it's useful, I work on a project where we maintain a repository of XPath selectors for extracting article content from many different sites: https://github.com/fivefilters/ftr-site-config - they're based on the original public Instapaper rules.

We also have PDF generation, but it's not really for crawling, and wasn't created for reading on a device like the Supernote, more for printing and reading: https://pdf.fivefilters.org/simple-print/

To accomplish full-text I ended up purchasing a license for https://www.fivefilters.org/full-text-rss/, self host it and bounce it through a docker container running Tor+privproxy which generates a new circuit every 10 minutes to help avoid IP based limits on certain websites I subscribe to. I can also disable the Tor bounce per-feed if needed.

Thanks for mentioning Instant View, I hadn't come across that. We actually maintain something similar here: https://github.com/fivefilters/ftr-site-config

We use these in our own tools and also get contributions from others, including Wallabag users: https://github.com/wallabag/wallabag

Before it was sold, Instapaper used to have something similar. A public database of its site-specific extraction templates. We used that as the starting point for our repository.

If you're trying to build one yourself, have a look at the open source Readability code[1]. It was originally developed by Arc90 and is now used by Apple and Mozilla in their browser reader views. The code has been ported to a number of different languages.

I work on a service called Full-Text RSS[2] that used a PHP port of Readability, coupled with site-specific extraction rules[3] to identify and extract article content from each feed item. It then produces a full-text version of the given feed. The idea is you subscribe to the full-text version in whichever feed reader you use and it will transparently give you full-text articles where you had partial content before.

[1] https://github.com/mozilla/readability

[2] https://www.fivefilters.org/full-text-rss/

[3] https://github.com/fivefilters/ftr-site-config

For those wondering if there's a redability lib in their favorite language. Here's a list of them all (as far as i know) plus the original arc-90 implementation

https://github.com/masukomi/arc90-readability/#readability

Please submit a PR if there's something i don't have listed there.

Huh, you are correct. I guess a better way to put this is "the original Readability I encountered was in Python"! The first version I saw was in Aaron Swartz's 2012 read2text tool, but a check of the URL I found that through says, yup, it's a Python port of Arc90's original code, which was a browser extension.

And you're right. It was in JavaScript. I finally tracked a copy down (the original is long evaporated): https://github.com/masukomi/arc90-readability/blob/master/js...