isa-l VS pixz

For using accelerators (QAT, I assume, because the article is about this) from the CLI, then QATzip[3] comes with a command line tool qatzip which can be used as described in the project ReadMe. I didn't test it, though, as I have no QAT-enabled device.

[1] https://github.com/intel/isa-l/tree/master/igzip

Ex 1: https://github.com/intel/isa-l/blob/master/README.md

Examples: - Someone I know created machine-optimized library in c and asm to accelerate compression, erasure coding, and encryption on Intel Xeon CPUs (https://github.com/intel/isa-l) which runs those ops WAY faster than the standard linux libraries. - My friend is making a retro SEGA genesis game using a C and asm dev kit (https://github.com/Stephane-D/SGDK)

igzip (https://github.com/intel/isa-l) is much faster than gzip or pigz when it comes to decompression, 2-3x in my experience. There is also a Python module (isal) that provides a GzipFile-like wrapper class, for an easy speed-up of Python scripts that read gzipped files.

However, it only supports up to level 3 when compressing data, so it can't be used as a drop-in replacement for gzip. You also need to make sure to use the latest version if you are going to use it in the context of bioinformatics, since older versions choke on concatenated gzip files common in that field.

That's really confusing since `pixz` exists and its "pixie" pronunciation actually works

https://github.com/vasi/pixz

pixz (https://github.com/vasi/pixz) is a nice parallel xz that additionally creates an index of tar files so you can decompress individual files. I wonder if dpkg could be extended to do something similar.

There's pixz, which indexes the tarball, allowing listing/extracting individual paths without decompressing the whole thing.

Also relevant is pixz [1] which can do parallel LZMA/XZ decompression as well as tar file indexing.

[1] https://github.com/vasi/pixz

Yes, it's as easy as installing pixz with symlinks pointing to xz (I think Debian even does this automatically as part of its post-installation scripts).

pixz - Parallel, indexed xz compressor. BSD-2-Clause

Not quite what you're asking, but if you're a 7-Zip fan and on Linux, you might be interested in pixz.