smaz VS FiniteStateEntropy

Choose the data compression algorithm based on the specifics of your data. For example, if you are working with lots of short strings, take a look at [*SMAZ](https://github.com/antirez/smaz).*

Take a look at this. Idea behind it seems nice, but it's fixed dictionary ("codebook") was clearly made for English language, and the algorithm itself is really simple. How can we impove on this? Dynamic dictionary won't do, since you have to store it somewhere, nullifying benefits of using such algorithm. Beyond that I have no idea.

smaz - Efficient string compression library. BSD-3-Clause

It's obsolete. It's limited to 32KB LZ window with huffman coding. Zstd can use a much larger window (8MB recommended) and a much better entropy coder: https://github.com/Cyan4973/FiniteStateEntropy

tANS block based : FSE

Sounds similar to: https://github.com/Cyan4973/FiniteStateEntropy

https://arxiv.org/abs/1311.2540

> The modern data compression is mainly based on two approaches to entropy coding: Huffman (HC) and arithmetic/range coding (AC). The former is much faster, but approximates probabilities with powers of 2, usually leading to relatively low compression rates. The latter uses nearly exact probabilities - easily approaching theoretical compression rate limit (Shannon entropy), but at cost of much larger computational cost.

FiniteStateEntropy - Two highly efficient compression codecs optimized for modern CPUs. BSD-2-Clause