Readability4J
soup-strainer
Readability4J | soup-strainer | |
---|---|---|
3 | 1 | |
135 | 33 | |
- | - | |
4.3 | 10.0 | |
over 2 years ago | about 10 years ago | |
HTML | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Readability4J
-
Creating an advanced search engine with PostgreSQL
Depending upon the type of content, one might want to look into using the Readability (Browder's reader view) to parse the webpage. It will give you all the useful info without the junk. Then you can put it in the DB as needed.
https://github.com/mozilla/readability
Btw, readability, is also available in few other languages like Kotlin:
https://github.com/dankito/Readability4J
-
How does Firefox's Reader View work?
My Hacker News client HACK for iOS and Android has a reader mode ability browser. While on iOS, I was able to use the reader mode feature provided by SFSafariViewController, that wasn't available on android.
So I had to read a ton about this. I ended up using a heavily modified Kotlin version of Readability:
https://github.com/dankito/Readability4J
https://play.google.com/store/apps/details?id=com.pranapps.h...
https://apps.apple.com/us/app/id1464477788
-
Show HN: Instantly Listen to Any URL
Not sure about OP but I just implemented this in my Hacker News android client (thanks for the idea OP).
This is how I implemented it. I had already achieved article to "reader mode" by heavily customizing the Kotlin port of Mozilla‘s Readability:
https://github.com/dankito/Readability4J
Then I pass the text via Android's TextToSpeech library and it works very well:
fun trySpeaking(str:String){
soup-strainer
-
How does Firefox's Reader View work?
I implemented a variation of the Readability algorithm some 9 years ago, in case anyone needs a server-side Python version and is interested in dragging it (kicking and screaming) into the 2020s:
https://github.com/rcarmo/soup-strainer
What are some alternatives?
go-readability - Go package that cleans a HTML page for better readability.
dom-distiller - Distills the DOM
article-extractor - To extract main article from given URL with Node.js
unclutter - A modern reader mode and article library for your browser.
Just-Read - A customizable read mode web extension.
trafilatura - Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
percollate - A command-line tool to turn web pages into readable PDF, EPUB, HTML, or Markdown docs.
go-trafilatura - go-trafilatura is a Go port of the trafilatura Python library.
web-clipper - For Notion,OneNote,Bear,Yuque,Joplin。Clip anything to anywhere
go-domdistiller - Go-DomDistiller is a Go port of the DOM Distiller library which implements Reader mode in Chrome for Android and Desktop. It has no dependencies on Chromium and is meant to run as a command line program or on a server.
go-htmldate - CLI and Go package for extracting publication date of a web pages.