ttrss_plugin-feediron VS mlscraper

It Technically could but as far as I am aware it doesn't.

Some alternatives

TT-RSS has multiple plugins, including the one I currently support https://github.com/feediron/ttrss_plugin-feediron

FressRSS has CSS selector support out of the box and has a readability extension that supports Readability or Mercury

I've been dreaming of porting Feediron to both FreshRSS and Nextcloud news. But I barely have any free time as is... one day

Back when I was using Tiny Tiny RSS I've developed af_feedmod to download the article from the linked webpage so you'd end up with a full feed. This was later forked into FeedIron and seems to be somewhat popular by now.

I'm a die hard TT-RSS user, mainly because of the Feediron plugin (A full text page parser) that I now maintain.

The official plugin uses a php port of Mozilla's Readability, which is used for Firefox Reader Mode. There is also the 3rd party FeedIron that is more configurable.

https://github.com/feediron/ttrss_plugin-feediron

Always good to see RSS projects pop up on hackernews. I'm still maintaining the Feediron plugin for TT-RSS - https://github.com/feediron/ttrss_plugin-feediron

Unlike this project Feediron is only for modifying existing RSS feeds to extract the desired information. Typically uses xpaths to select content

> I've never open sourced it though because I guess it's a bit of a grey area

I'm maintaining the TT-RSS plugin feediron https://github.com/feediron/ttrss_plugin-feediron that fetches full-text data, so my thinking is this:

At the end of the day if it's a openly available website and you are personally (through your own server) fetching the resources I don't think anyone has a right to complain.

Now if you were offering it as a service it might arguably be a bit more grey, but only if you're ignoring the robots.txt file

So you plan something like the FeedIron TT-RSS Plugin (https://github.com/feediron/ttrss_plugin-feediron) to allow customization of a feed to get relevant content?

See if something like autoscraper or mlscraper suits your needs.

Why GPT-based then? There are libraries that do this: You give examples, they generate the rules for you and give you a scraper object that takes any html and returns the scraped data.

Mine: https://github.com/lorey/mlscraper

GitHub - lorey/mlscraper: 🤖 Scrape data from HTML websites automatically by just providing examples

Check it out here: https://github.com/lorey/mlscraper Example: https://github.com/lorey/mlscraper/blob/master/examples/quotes\_to\_scrape.py

In case anyone wants to detect the selectors automatically, here's a small python library I wrote that does it for you: https://github.com/lorey/mlscraper