parser VS hn-search

Thoroughly scraping is challenging, especially in an environment where you don’t have (or want) a JavaScript runtime.

For content extraction, I found the approach the Postlight library takes quite neat. It scores individual html nodes based on some heuristics (text length, link density, css classes). It the selects the nodes with the highest score. [1] I ported it to Swift for a personal read later app.

[1] https://github.com/postlight/parser

I've been working on an enhanced reader mode extension for the last few months. I found that Mercury Reader's parser tool is useful for extracting content. If that's not exactly what you're looking for, readibility is another good option. It's a library used inside Firefox's reader moder that you can use in any project.

I currently have it turned off while I search for better sources, but I have a VM that runs a custom cron script that combines a custom RSS reader, podfox, mercury-parser, and coqui-ai to generate audio podcasts from RSS news feeds. I should probably clean it up and release the script/setup process. With a few tweaks and some AI text-to-speech and a little machine learning audio processing you can get a really good podcast experience from text posts.

It looks like Relay could be updated to convert it locally though, since the parser that it uses appears to be open source.

https://github.com/postlight/mercury-parser The only one I need, shit's too good

An alternative would be to extract the plain text from the article's page with either some "readability" API or a library like Mercury Parser: https://github.com/postlight/mercury-parser

I haven’t directly compared them, but I have also found mercury parser (https://github.com/postlight/mercury-parser) to be very reliable.

Since it turns a website into very plain (X)HTML it‘s fairly easy to use it to make a browsing proxy or automatically produce epub files for e-readers, which is what I do.

Make sure that at the end of the process you have the node and npm executables installed - the http.webpage integration uses the Mercury Parser API to convert web pages to Markdown.

Good luck! Those HTML issues you're coming across are tough and so varied across the web!

I was working with Mercury Parser (pluggable parsing for different sites) in the past.

https://github.com/postlight/mercury-parser

The submitted title was "Gary Killdall, creator of CP/M, wrote Pixar's original 3D renderer".

Submitters: If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...

(From https://news.ycombinator.com/newsguidelines.html: "Please use the original title, unless it is misleading or linkbait; don't editorialize.")

Vision therapy for myopia helps some people, but not everyone, likely due to genetic and neuroplasticity differences, https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Nevertheless, many of the principles are useful for children whose eyes and brains are still developing.

I'm a huge Tesla skeptic, but Tesla and Musk are lightning rods for tabloid-style garbage that doesn't belong on HN, so it doesn't surprise me that we often see negative Tesla content flagged to death. Meanwhile we also see plenty of content that hits the front page and stays there [0].

Do you have examples of professional, interesting Tesla content that got flagged?

[0] More than half of the past year's most popular Tesla articles were negative: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...

It's April 23rd, 2024, and I am still looking for a good, reliable, honest and simple search engine.

All I want to do is search.

No AI.

No ads.

No shopping.

Please don't "Answer my question." I enjoy doing my own original research, thanks.

I'm entirely willing - wanting even - to pay for it.

Currently Kagi has my $, but I'm saddened and frustrated that they're not even focused on Search, they're focused on AI[1] and t-shirts.

Amazingly, in 2024, there is still a market opportunity for a good search engine.

It can't really just be me, can it?

[1]: https://hn.algolia.com/?query=%22kagi%22+%22ai%22

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

For historical purposes

Most likely because there have been oodles of low-quality stories on these topics. We turned the flags off on this one since it maybe rises above the noise (see https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... for past explanations on how we approach that).

Hey HN,

I'm sure you've seen the monthly "Ask HN: What Are You Working On?" headlines on [Hacker News](https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...).

Honestly, it's my favorite topic because it's packed with insights about what other hackers are up to.

I wondered what it would be like if instead of just a headline, there was a whole website where hackers could post daily updates, and where we could follow the hackers we're interested in for their latest updates. And so, this web site was born.

I hope it gets used frequently so we can all benefit from it together. I look forward to hearing your thoughts.

Let me know what you think!

I don't know what one thing you're referring to, but it's a core principle of HN to try to avoid repetition, and especially the repetition+indignation combo, which is the commonest and most tedious thing on the internet.

https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...

Happy 10,000 day to you

https://news.ycombinator.com/from?site=nand2tetris.org

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

not sure. I searched comments: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=fal...

Most recent are more culture wars stuff but some earlier ones appear to suggesting a degree of alignment with the USA government.