isa-l VS zip.js

For using accelerators (QAT, I assume, because the article is about this) from the CLI, then QATzip[3] comes with a command line tool qatzip which can be used as described in the project ReadMe. I didn't test it, though, as I have no QAT-enabled device.

[1] https://github.com/intel/isa-l/tree/master/igzip

Ex 1: https://github.com/intel/isa-l/blob/master/README.md

Examples: - Someone I know created machine-optimized library in c and asm to accelerate compression, erasure coding, and encryption on Intel Xeon CPUs (https://github.com/intel/isa-l) which runs those ops WAY faster than the standard linux libraries. - My friend is making a retro SEGA genesis game using a C and asm dev kit (https://github.com/Stephane-D/SGDK)

igzip (https://github.com/intel/isa-l) is much faster than gzip or pigz when it comes to decompression, 2-3x in my experience. There is also a Python module (isal) that provides a GzipFile-like wrapper class, for an easy speed-up of Python scripts that read gzipped files.

However, it only supports up to level 3 when compressing data, so it can't be used as a drop-in replacement for gzip. You also need to make sure to use the latest version if you are going to use it in the context of bioinformatics, since older versions choke on concatenated gzip files common in that field.

Similarly, if people are interested, I have coded the possibility to compress zip files on several cores in zip.js [1]. The approach is simpler as it consists of compressing the entries in parallel. It still offers a significant performance gain though when compressing multiple files in a zip file, which is often the nominal case.

[1] https://github.com/gildas-lormeau/zip.js

This shouldn't be an issue. There are JS libraries that can decompress zip (e.g. https://gildas-lormeau.github.io/zip.js/). Nowadays even huge C/C++ codebases can be compiled into JS via Emscripten.

Well, you can build zip.js with fflate if you want to, see https://github.com/gildas-lormeau/zip.js/blob/master/rollup-fflate.config.js. I wasn't saying that zip.js is faster than fflate or any other library. I'm just saying it can compress files in parallel.