ThreadPool VS express-c

And in C++ can also use this dead-simple header file for a nice high-level, modern threadpool using function objects (lambdas) for very easy parallelization of arbitrary tasks: https://github.com/progschj/ThreadPool

Depending on how sophisticated you are with C++ you might want to try to impress him by using multithreading. I like this simple library. It's only like a 100 lines of code in a single header, and it is very simple to use.

#!/usr/bin/env bash if [ ! -d kenlm ]; then git clone https://github.com/kpu/kenlm.git echo -e "\n" fi if [ ! -d openfst-1.6.3 ]; then echo "Download and extract openfst ..." wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.6.3.tar.gz tar -xzvf openfst-1.6.3.tar.gz echo -e "\n" fi if [ ! -d ThreadPool ]; then git clone https://github.com/progschj/ThreadPool.git echo -e "\n" fi echo "Install decoders ..." python setup.py install --num_processes 4

For point 2, just use this if you're lazy: https://github.com/progschj/ThreadPool

GCD/libdispatch is a fantastic approach to concurrency and you can build and install support for non-Apple operating systems:

https://github.com/apple/swift-corelibs-libdispatch

Here’s a simple echo server:

https://github.com/williamcotton/c_playground/blob/master/sr...

Here’s a simple multithreaded database pool:

https://github.com/williamcotton/express-c/blob/master/src/d...

Apache does this! And I do this in my own C web framework:

https://github.com/williamcotton/express-c/blob/master/deps/...

Unfortunately not all of the sanitizers are working in clang for Apple Silicon machines. For my own CI pipeline I run both Darwin and Linux builds, leaks and valgrind respectively, along with the kitchen sink of compiler sanitizers:

Here’s the GitHub actions for those platforms and a link to the Makefile:

GitHub Action:

https://github.com/williamcotton/express-c/blob/master/.gith...

Makefile:

https://github.com/williamcotton/express-c/blob/f2e1dde2f5a7...

Lol, yeah, the libdispatch runtime is probably about as big as the Go runtime!

I think this has a slight advantage in that it uses a memory arena for a per-request bump allocator so it should keep overall memory usage lower.

The slowest part of all this has to do with Block_copy and the places where I’m very much treating blocks/closures in an OOP type manner.

This could be fixed by writing a different function that did basically the same thing as Block_copy but for all the “methods” on an “object” in a single pass. But who has time for that? :D

Here’s more about those performance issues and some text code that show how slow the approach is:

https://github.com/williamcotton/express-c/tree/master/resea...

I’ve used an approach to copy only the shared libs from the build container to the production container. This not only gets rid of most of this “dark matter” but results in much smaller containers!

https://github.com/williamcotton/express-c/blob/master/demo/...

I use arenas for HTTP request handlers. The JSON built or parsed is only going to be around for the lifetime of the request. It's much simpler and faster to use an arena than to do reference counting wrapped around malloc/free.

Here's an example where in a commit I swapped in an arena:

https://github.com/williamcotton/express-c/commit/4ae53f38e3...