piscina VS worm-scraper

Workers multi-threading via Tinypool (a lightweight fork of Piscina)

For threading best to use piscina and for clustering best to use pm2 which does not even need one line of code change in Node.js code.

We used piscina at it was quite good: https://github.com/piscinajs/piscina

We used Benchmark.js for benchmarking and piscina as a pool for worker threads. benchmark.js was used to run the same code in 2 scenarios - one using a single thread and one using the piscina pool. The degree of parallelism was passed to the program via an environment variable. The test code is present in worker.js in both the cases.

piscina for managing node's worker threads

One other thing...this passed by in my feeds last week, I’m not an expert in web workers or browser threads etc, but it seems like it might be relevant, so sharing it just in case:

“A fast, efficient Node.js Worker Thread Pool implementation”

https://github.com/piscinajs/piscina

This is by no means a complete list of what piscina can do.... checkout the official docs here.

If you (or someone you know) is technical, here is a program that creates an e-book for your personal use based on the HTML pages. That's how I ended up reading it: https://github.com/domenic/worm-scraper

https://github.com/domenic/worm-scraper can be adapted for any site.

I maintain a command-line utility which assembles an eBook from scraped .xhtml files. It uses a similar package, workerpool, to process multiple chapters in parallel.

https://github.com/domenic/worm-scraper/blob/master/lib/conv...

Officially is only available in its website. But you can scrap a epub version yourself with this extremely easy to use tool and then convert to kindle format.

You can use this tool to produce an ebook of Worm and read it with your favorite ebook reader. Just make sure not to distribute the ebook itself anywhere on the internet.