selectolax VS lambdasoup

https://github.com/rushter/selectolax#simple-benchmark )

(Apache Nutch is a Java-based web crawler which supports e.g. CommonCrawl (which backs various foundational LLMs)) https://en.wikipedia.org/wiki/Apache_Nutch#Search_engines_bu... . But extruct extracts more types of metadata and data than Nutch AFAIU: https://github.com/scrapinghub/extruct )

datasette-graphql adds a GraphQL HTTP API to a SQLite database:

selectolax

try this... https://github.com/rushter/selectolax

Lazyweb link: https://github.com/rushter/selectolax

although I don't follow the need to have what appears to be two completely separate HTML parsing C libraries as dependencies; seeing this in the readme for Modest gives me the shivers because lxml has _seen some shit_

> Modest is a fast HTML renderer implemented as a pure C99 library with no outside dependencies.

although its other dep seems much more cognizant about the HTML5 standard, for whatever that's worth: https://github.com/lexbor/lexbor#lexbor

---

> It looks like the author of the article just googled some libraries for each language and didn't research the topic

Heh, oh, new to the Internet, are you? :-D

I'm using soupault right now to make a simple company wiki (under a dozen pages). I like how it's HTML-first and easily customizable, compared to other static site generators that come with too many bells and whistles. Although now I have to make my own image compression script...

Also, fun fact: soupault is written in OCaml, which apparently has a really nice library for HTML manipulation: https://github.com/aantron/lambdasoup

OCaml’s Lambda Soup (https://aantron.github.io/lambdasoup/) is a amazing library/, especially for those that prefer functional programming

It's used for sorting "widgets" (page processing steps) according to dependency lists that users can specify in the config (like `after = ["foo", "bar"]`).

Other than that, one thing I really like about OCaml is that the compiler team and most library maintainers are considerate towards downstream users with respect to compatibility.

The Lua interpreter [3] that soupault uses for its plugin API is a revived 20 year old research project. It only needed minor modifications to build with recent compiler versions.

[1] https://github.com/aantron/lambdasoup