lol-html VS hn-search

So I'm doing some web scraping in Rust, and so I will need to parse HTML. [scraper](https://docs.rs/scraper/latest/scraper/) (which uses [html5ever](https://github.com/servo/html5ever)) is doing fine except that it's the bottleneck of my application.

So I need a faster parser. I've tried [tl](https://docs.rs/tl/latest/tl/) which would've been perfect except that it doesn't actually work on the HTML I have. When I try to `query_selector` the elements I need, it returns nothing.

[Kuchiki](https://docs.rs/kuchiki/latest/kuchiki/) is abandonded.

I couldn't figure out how to get [lol-html](https://github.com/cloudflare/lol-html) to work for me (it's designed for re-writing HTML, whatever that means). It doesn't seem to have an API to extract the inner text of an element.

[html5gum](https://github.com/untitaker/html5gum) seems to be just an HTML tokenizer, or otherwise just too low-level. I have not yet tried [quick-xml](https://github.com/tafia/quick-xml/) but judging from the README, it's pretty low-level too. I mean, if these are the only options left then I will try them. Otherwise, I would love to use a parser that's faster but as ergonomic as `scraper` or `tl`.

At this point, I would be happy with an Lxml bridge/port of some sort. I don't need to mutate HTML, just parse and read data from it.

I'm also in the Workers org but I have had a bit of interaction with Rust. There's some Rust in the Workers runtime using lol-html for HTMLRewriter as well as some tooling and there's the full blown workers-rs framework that I work on, but that's about it for the Rust I work on regularly.

Thanks again and happy to answer any questions! My current unreleased Rust projects include a web crawler that uses Tokio + Tokio Console + Reqwest with this crate for robots.txt and a fast text extraction library using lol-html that I am planning to sprinkle with some minimal ML to get Readability.js style intelligent extraction (with training in Python). See Fathom for an example of the ML approach I'll likely take.

I’d like to see a tool using lol-html [0] and their CSS selector API as a streaming HTML editor.

[0] https://github.com/cloudflare/lol-html

Cloudflare has open sourced lol-html, a "Low output latency streaming HTML parser/rewriter with CSS selector-based API". Is that what you are looking for?

Vision therapy for myopia helps some people, but not everyone, likely due to genetic and neuroplasticity differences, https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Nevertheless, many of the principles are useful for children whose eyes and brains are still developing.

I'm a huge Tesla skeptic, but Tesla and Musk are lightning rods for tabloid-style garbage that doesn't belong on HN, so it doesn't surprise me that we often see negative Tesla content flagged to death. Meanwhile we also see plenty of content that hits the front page and stays there [0].

Do you have examples of professional, interesting Tesla content that got flagged?

[0] More than half of the past year's most popular Tesla articles were negative: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...

It's April 23rd, 2024, and I am still looking for a good, reliable, honest and simple search engine.

All I want to do is search.

No AI.

No ads.

No shopping.

Please don't "Answer my question." I enjoy doing my own original research, thanks.

I'm entirely willing - wanting even - to pay for it.

Currently Kagi has my $, but I'm saddened and frustrated that they're not even focused on Search, they're focused on AI[1] and t-shirts.

Amazingly, in 2024, there is still a market opportunity for a good search engine.

It can't really just be me, can it?

[1]: https://hn.algolia.com/?query=%22kagi%22+%22ai%22

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

For historical purposes

Most likely because there have been oodles of low-quality stories on these topics. We turned the flags off on this one since it maybe rises above the noise (see https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... for past explanations on how we approach that).

Hey HN,

I'm sure you've seen the monthly "Ask HN: What Are You Working On?" headlines on [Hacker News](https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...).

Honestly, it's my favorite topic because it's packed with insights about what other hackers are up to.

I wondered what it would be like if instead of just a headline, there was a whole website where hackers could post daily updates, and where we could follow the hackers we're interested in for their latest updates. And so, this web site was born.

I hope it gets used frequently so we can all benefit from it together. I look forward to hearing your thoughts.

Let me know what you think!

I don't know what one thing you're referring to, but it's a core principle of HN to try to avoid repetition, and especially the repetition+indignation combo, which is the commonest and most tedious thing on the internet.

https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...

Happy 10,000 day to you

https://news.ycombinator.com/from?site=nand2tetris.org

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

not sure. I searched comments: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=fal...

Most recent are more culture wars stuff but some earlier ones appear to suggesting a degree of alignment with the USA government.

This is a topic that has come up a ton on HN, in submitted articles[1] and practically once a day in comments[2]

[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...