smaz VS Snappy

Choose the data compression algorithm based on the specifics of your data. For example, if you are working with lots of short strings, take a look at [*SMAZ](https://github.com/antirez/smaz).*

Take a look at this. Idea behind it seems nice, but it's fixed dictionary ("codebook") was clearly made for English language, and the algorithm itself is really simple. How can we impove on this? Dynamic dictionary won't do, since you have to store it somewhere, nullifying benefits of using such algorithm. Beyond that I have no idea.

smaz - Efficient string compression library. BSD-3-Clause

Another example of Nim being really fast is the supersnappy library. This library benchmarks faster than Google’s C or C++ Snappy implementation.

It doesn't destroy performance for the simple reason that nowadays memory access has higher latency than pure compute. If you need to use compute to produce some data to be stored in memory, your overall throughput could very well be faster than without compression.

There have been a large amount of innovation on fast compression in recent years. Traditional compression tools like gzip or xz are geared towards higher compression ratio, but memory compression tends to favor speed. Check out those algorithms:

* lz4: https://lz4.github.io/lz4/

* Google's snappy: https://github.com/google/snappy

* Facebook's zstd in fast mode: http://facebook.github.io/zstd/#benchmarks

Google released Snappy, which is extremely fast and robust (both at compression and decompression), but it's definitely not nearly as good (in terms of compression ratio). Google mostly uses it for real-time compression, for example of network messages - not for long-term storage.

Just compress it! Of course if you will you ZIP, players will able to just open this zip file and change whatever they want. But you can use less popular compression algorithms which are not supported by default Windows File Explorer. Snappy for example.