ftr-site-config VS fraidycat

As far as full-text caching... maybe a self-hosted instance or paid version of the FiveFilters Full-Text RSS service would work. You can integrate that into whatever aggregator you want.

However you can retrofit this onto any reader by using a service that creates a full text feed from a summary feed. Two that I have used in the past are https://morss.it/ and https://www.fivefilters.org/full-text-rss/.

RSS feeds that don't contain the full article text drive me nuts.

Here is a workaround that I've had good luck with:

https://www.fivefilters.org/full-text-rss/

In addition to improving usability, it defeats attempts to measure clickbait summary efficacy, etc., since it breaks sites' ability to pull popularity / telemetry info.

By any chance, could this be used as an alternative to the full-article RSS tool that FiveFilters offers?

Please check out FullTextRSS from Five Filters: https://www.fivefilters.org/full-text-rss/

They have an OSS version you can host yourself. It fixes the problem of sites not sharing their full text in their feed, by going and scraping the site into a full feed for you.

Full-Text RSS - FiveFilters.org

This is great!

If it's useful, I work on a project where we maintain a repository of XPath selectors for extracting article content from many different sites: https://github.com/fivefilters/ftr-site-config - they're based on the original public Instapaper rules.

We also have PDF generation, but it's not really for crawling, and wasn't created for reading on a device like the Supernote, more for printing and reading: https://pdf.fivefilters.org/simple-print/

To accomplish full-text I ended up purchasing a license for https://www.fivefilters.org/full-text-rss/, self host it and bounce it through a docker container running Tor+privproxy which generates a new circuit every 10 minutes to help avoid IP based limits on certain websites I subscribe to. I can also disable the Tor bounce per-feed if needed.

Thanks for mentioning Instant View, I hadn't come across that. We actually maintain something similar here: https://github.com/fivefilters/ftr-site-config

We use these in our own tools and also get contributions from others, including Wallabag users: https://github.com/wallabag/wallabag

Before it was sold, Instapaper used to have something similar. A public database of its site-specific extraction templates. We used that as the starting point for our repository.

If you're trying to build one yourself, have a look at the open source Readability code[1]. It was originally developed by Arc90 and is now used by Apple and Mozilla in their browser reader views. The code has been ported to a number of different languages.

I work on a service called Full-Text RSS[2] that used a PHP port of Readability, coupled with site-specific extraction rules[3] to identify and extract article content from each feed item. It then produces a full-text version of the given feed. The idea is you subscribe to the full-text version in whichever feed reader you use and it will transparently give you full-text articles where you had partial content before.

[1] https://github.com/mozilla/readability

[2] https://www.fivefilters.org/full-text-rss/

[3] https://github.com/fivefilters/ftr-site-config

Fraidycat looks good, though it's more like a client than a self-hosted service. I especially like that it can follow / aggregate a bunch of different types of sources. https://github.com/kickscondor/fraidycat

Wonder what's are the differences from https://github.com/DIYgod/RSSHub and https://github.com/kickscondor/fraidycat

I guess projects might have different goals in mind, but IMO would be nice to share the scrapers among the projects

If you're interested in furthering RSS (and Atom, JSON Feed) - one way to help this along is to continue extending it to bring it modern. For instance, I've documented an extension for adding temporary 'status' to the feed - useful for pinned posts or broadcasting that a livestream is beginning.

https://github.com/kickscondor/fraidycat/wiki/RSS-Atom-Exten...

It feels like there is still a lot of room to extend these formats - part of the advantage to them is how easily they can be extended. The media enclosure extension is the reason they've been so useful for podcast subscriptions.

And very down to earth. He recently took time to track down a bug in Fraidycat:

https://github.com/kickscondor/fraidycat/issues/220

fraidycat I dont know if it will help or not.