threads.js VS piscina

Threads.js makes it pretty easy to spawn a thread pool and handle multi-threading workloads.

We did this via the npm threads wrapper and esbuild. We first moved all of our compute code into a new file with minimal imports, called imageCompare.worker.js. We then added a pre-compilation step with esbuild to compile this file into a bundle. We then spawn the worker using this generated file as a blob, and interact with it via the threads promise interface.

That latter one is in it's own process, and I also use multiple threads with the threads library.

This app has a memory leak where your worker threads are spawning out of control. You can verify this in Chrome DevTools Source tab: after you click the Generate button, you can see the worker threads spawning out of control until you close the process.

I'd advise that you do a better job at manually spawning and terminating the threads with the TFJS process. I've had success using this library in combination with TFJS: https://threads.js.org/

You can use it for everything. Even the compute tasks you can offload to worker threads using something like https://github.com/andywer/threads.js or you could create intensive modules in Rust and integrate using something like https://github.com/neon-bindings/neon

Web worker meets worker threads – threads.js\ (3 comments)

threads.js

Workers multi-threading via Tinypool (a lightweight fork of Piscina)

For threading best to use piscina and for clustering best to use pm2 which does not even need one line of code change in Node.js code.

We used piscina at it was quite good: https://github.com/piscinajs/piscina

We used Benchmark.js for benchmarking and piscina as a pool for worker threads. benchmark.js was used to run the same code in 2 scenarios - one using a single thread and one using the piscina pool. The degree of parallelism was passed to the program via an environment variable. The test code is present in worker.js in both the cases.

piscina for managing node's worker threads

One other thing...this passed by in my feeds last week, I’m not an expert in web workers or browser threads etc, but it seems like it might be relevant, so sharing it just in case:

“A fast, efficient Node.js Worker Thread Pool implementation”

https://github.com/piscinajs/piscina

This is by no means a complete list of what piscina can do.... checkout the official docs here.