lexbor VS gumbo-parser

Project is deprecated in favour of the same developer's lexbor project[0].

[0]: https://github.com/lexbor/lexbor

It supports both CSS selectors and XPath like Nokogiri, but with separate engines - parsing and CSS engine by Lexbor, XPath engine by libxml2. (Nokogiri internally converts CSS selectors to XPath syntax, and uses XPath engine for all searches).

An HTML parser, probably the simplest relatively modern example I could find is 1MB https://github.com/lexbor/lexbor (haven't used it, but might look more into it now that I know it exists.)

Lazyweb link: https://github.com/rushter/selectolax

although I don't follow the need to have what appears to be two completely separate HTML parsing C libraries as dependencies; seeing this in the readme for Modest gives me the shivers because lxml has _seen some shit_

> Modest is a fast HTML renderer implemented as a pure C99 library with no outside dependencies.

although its other dep seems much more cognizant about the HTML5 standard, for whatever that's worth: https://github.com/lexbor/lexbor#lexbor

---

> It looks like the author of the article just googled some libraries for each language and didn't research the topic

Heh, oh, new to the Internet, are you? :-D

Lexbor is here: https://github.com/lexbor/lexbor

Well, regarding HTML5, what I've found was libxml (does not support tag-soup HTML5), https://github.com/lexbor/lexbor, for which I was unable to find good documentation ( see https://lexbor.com/docs/lexbor/#dom), Apache Xerces (appears to not support tag-soup HTML5 as well), and Gumbo, which does not appear to be active and to support selectors and XPath (although there are libraries that add that).

I think we've all (mostly?) tried it. It really is the Wild West of the web when you're trying to parse other people's HTML, though.

I've played around with this parser which is extremely quick. https://github.com/lexbor/lexbor

I’m glad to have the opportunity to contribute to an open-source project that is used by thousands of people. Hopefully, we will speed up Nokogiri (or XML parser it uses) to match the performance of html5ever or lexbor at some point in the future. 800 ms to extract data from HTML is still too much.

It uses libcurl and gumbo (https://github.com/google/gumbo-parser). Gumbo is apparently written in pure C99 (interestingly Curl is written in the even older C89 standard). Will've been more amusing if article was written considering that and used C99.

> A standards compliant HTML5 parser is at the bare minimum millions of lines of code.

But https://github.com/google/gumbo-parser is only 34K lines?

oops... I saw a markup parser and automatically thought XML, but you are right! HTML is actually a whole different beast!

As it turns out, seems like nim also has an html parser [1], but I'm guessing something like Google's gumbo [2] could be more reliable, but you would have to write bindings for nim.

1: https://nim-lang.org/docs/htmlparser.html

2: https://github.com/google/gumbo-parser

Well, regarding HTML5, what I've found was libxml (does not support tag-soup HTML5), https://github.com/lexbor/lexbor, for which I was unable to find good documentation ( see https://lexbor.com/docs/lexbor/#dom), Apache Xerces (appears to not support tag-soup HTML5 as well), and Gumbo, which does not appear to be active and to support selectors and XPath (although there are libraries that add that).

I haven't used any of them, but there's a few wrappers available for Gumbo.