lz4_flex VS 7-Zip-zstd

I run a benchmark based on my LZ4 implementation (de/compressor) between cax11 and cx11 since the instances cost the same. The code doesn't use any SIMD.

By the way, the PR removing bounds checks from extend_from_within_overlapping is up: https://github.com/PSeitz/lz4_flex/pull/141

Zap uses LZ4 as it's primary compression algorithm, although I might add ZSTD as an option too. The LZ4 crate i'm using goes over the compression better than I can in the span of a comment. Check it out here.

I wrote something here how hc could work: https://github.com/PSeitz/lz4_flex/issues/21

Yes, the updated benchmarks are here: https://github.com/pseitz/lz4_flex#results-v080-17-05-2021-safe-decode-and-safe-encode-off

I ported the block format to Rust matching the C implementation in performance and ratio.

https://github.com/pseitz/lz4_flex

Following this change count_same_bytes is unsound - it offsets the pointer by the value of candidate without any bounds checks, which may result in out-of-bounds access.

Do you consider rar a standard? It's a pretty good format, even though there aren't any good open-source implementations. But if you're willing to pay for your software, WinRAR command line versions are available for most platforms.

7zip is the most obvious free alternative. There is also a 7zip fork that offers zstd [1]. The command line experience for 7zip isn't very good however.

1: https://github.com/mcmilk/7-Zip-zstd

Check out https://github.com/mcmilk/7-Zip-zstd

There's also this fork which supports additional compressions: https://github.com/mcmilk/7-Zip-zstd

7-zip is actually not a bad option, but I'd strongly advise using the fast-lzma2 codec, because, well, it is waay faster, and might even give you better ratios, depending on how you use it. You can find an implementation here, and it is compatible with traditional 7-zip (you can decompress it on any computer with any newish version of 7z)

Hi, PeaZip can use McMilk's additional compression codecs for 7z format (Zstandard, Brotli, Lizard, etc), https://github.com/mcmilk/7-Zip-zstd

You can extract the files yourself with 7Zip. You can install 7Zip from here and then install this plugin to extract ZStandard files, or you can directly install the modified 7Zip with the plugin already from that plugin page. Then simply open the zst file you downloaded with 7Zip and extract it.