unclutter VS parser

no a filter for uBO (I do not think it is possible) but I really like this extension: https://github.com/lindylearn/unclutter

Other browsers can use the uncluttered extension.

There already is crowdsourcing of broken page reports: https://github.com/lindylearn/unclutter/issues?q=is%3Aissue+...

And twitter.com is a special case: https://github.com/lindylearn/unclutter/issues/570

I'm working on those, but it's never going to be perfect unfortunately.

Here’s more info: unclutter.lindylearn.io

I'm looking into this: https://github.com/lindylearn/unclutter/issues/661

It all started with a r/chrome post a few months ago, and since then we've added many improvements to our GitHub project.

Hey! This is a reader mode browser extension I built that hides noisy page elements rather than extracting and re-rendering only the page text. The idea is to not make all articles look the same [0], have them still render graphs, and ideally to work in more cases.

There are a few "tricks": patching the site CSSOM to apply simpler mobile styles even at desktop width, detecting the likely main text & removing its non-text siblings, blocklists for classnames that contain words like "sidebar", and testing this on a few hundred popular sites.

I got carried away and also added a dark mode, page outlines, private annotations & inline Hacker News comments.

The last feature works by parsing every top-level HN comment with a quote in it (formatted with > or "") within a few minutes, and anchoring these quotes in the related article HTML. So when you click a link on HN you’ll see the parts people are talking about while reading. [1]

The code is all on GitHub!

[0] Screenshots comparing it to the Firefox reader mode: https://github.com/lindylearn/unclutter/blob/main/docs/compa...

[1] It's fun to try this on some of the "HN classics" that got 30+ quote comments over the years. The list at hn.lindylearn.io/best shows the number of “annotations” an article has beneath its title.

You can also contribute to the existing extension for this: https://github.com/lindylearn/unclutter

More examples: unclutter.lindylearn.io The code: github.com/lindylearn/unclutter

Hey! This is a reader mode browser extension I built that hides noisy page elements rather than extracting and re-rendering only their text content. The idea is to not make all articles look the same [0], have them still render graphs, and ideally to work in more cases.

There are a few "tricks": patching the site CSSOM to apply simpler mobile styles even at desktop width, cleaning up parents of DOM text nodes, blocklists for class names that contain words like "sidebar", plus manual CSS patches for popular sites.

I got carried away and also added a dark mode, page outlines, privates notes & inline Hacker News comments. [1]

The last feature works by parsing every top-level HN comment with a quote in it (formatted with > or "") within a few minutes, and anchoring these quotes in the story article HTML. So when you open a link you'll directly see the parts people are talking about here. [2]

The extension code is all on GitHub: github.com/lindylearn/unclutter

[0] Unclutter vs the Firefox reader mode: https://github.com/lindylearn/unclutter/blob/main/docs/compa...

[1] The linked website show some examples for these.

[2] It's also fun to try this on some of the "HN classics" that got 30+ quote comments over the years. The list at hn.lindylearn.io/best shows the number of "annotations" a link has beneath its title.

Thoroughly scraping is challenging, especially in an environment where you don’t have (or want) a JavaScript runtime.

For content extraction, I found the approach the Postlight library takes quite neat. It scores individual html nodes based on some heuristics (text length, link density, css classes). It the selects the nodes with the highest score. [1] I ported it to Swift for a personal read later app.

[1] https://github.com/postlight/parser

I've been working on an enhanced reader mode extension for the last few months. I found that Mercury Reader's parser tool is useful for extracting content. If that's not exactly what you're looking for, readibility is another good option. It's a library used inside Firefox's reader moder that you can use in any project.

I currently have it turned off while I search for better sources, but I have a VM that runs a custom cron script that combines a custom RSS reader, podfox, mercury-parser, and coqui-ai to generate audio podcasts from RSS news feeds. I should probably clean it up and release the script/setup process. With a few tweaks and some AI text-to-speech and a little machine learning audio processing you can get a really good podcast experience from text posts.

It looks like Relay could be updated to convert it locally though, since the parser that it uses appears to be open source.

https://github.com/postlight/mercury-parser The only one I need, shit's too good

An alternative would be to extract the plain text from the article's page with either some "readability" API or a library like Mercury Parser: https://github.com/postlight/mercury-parser

I haven’t directly compared them, but I have also found mercury parser (https://github.com/postlight/mercury-parser) to be very reliable.

Since it turns a website into very plain (X)HTML it‘s fairly easy to use it to make a browsing proxy or automatically produce epub files for e-readers, which is what I do.

Make sure that at the end of the process you have the node and npm executables installed - the http.webpage integration uses the Mercury Parser API to convert web pages to Markdown.

Good luck! Those HTML issues you're coming across are tough and so varied across the web!

I was working with Mercury Parser (pluggable parsing for different sites) in the past.

https://github.com/postlight/mercury-parser