Zstandard - Fast real-time compression algorithm
LZ4 is so fast, that in make sense to use it everywhere over uncompressed data. Even storing items in-memory compressed sometimes is profitable as you can fit more items in memory.
Still zstd offers way better compression and got variable difficulty factor: https://github.com/facebook/zstd Decompression is always fast, but you can trade off compression vs. ratio factor.
In general if send data over network zstd is quite profitable. Even network attached disk AWS EBS or AWS S3 it can be a hugely profitable.
7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
Here’s a fork of the Windows compression tool 7-Zip which has LZ4 support baked in along with some other useful algorithms – the repo has a good comparison of them: https://github.com/mcmilk/7-Zip-zstd/
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
A new set of tools and libraries for working with SquashFS images
A while ago I did some simplistic SquashFS pack/unpack benchmarks. I was primarily interested in looking at the behavior of my thread-pool based packer, but as a side effect I got a comparison of compressor speed & ratios over the various available compressors for my Debian test image.
I must say that LZ4 definitely stands out for both compression and uncompression speed, while still being able to cut the data size in half, making it probably quite suitable for life filesystems and network protocols. Particularly interesting was also comparing Zstd and LZ4, the former being substantially slower, but at the same time achieving a compression ratio somewhere between zlib and xz, while beating both in time (in my benchmark at least).
Pure Rust implementation of raw LZ4 compression/decompression.
I ported the block format to Rust matching the C implementation in performance and ratio.
Superfast compression library
A library that provides an embeddable, persistent key-value store for fast storage.
W.r.t. standard dictionaries, it's something we're interested in, but the fundamental reality of dictionaries is that their effectiveness is strongly tied to their specificity. Put another way, a universal dictionary is a self-contradiction.
And yes, totally, I know at least RocksDB supports exactly that behavior .
Extremely Fast Compression algorithm
I'm not a fan of the stacked bar charts, I like the table of data for "Benchmarks" on the github source page: https://github.com/lz4/lz4
It makes it very clear where LZ4 fits into comparisons with compression speed, decompression speed and compression ratio
OPS - Build and Run Open Source Unikernels. Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
OpenZFS on Linux and FreeBSD
> zstd is now standard and the default on openzfs 2.0, replacing lz4.
Are you sure? The default compression level has always been "off", but when switched on - the default has been lz4 for about 5 years. Zstd support was added less than a year ago and there are still a lot of things that need to be fixed before one could even suggest that it might be a sane default. I like zstd, but I like my uncorrupted data more. I know that compatibility between compressor versions and pools is a concern, and there are also the compression performance problems with the way zstd handles zfs block sizes. Thankfully lz4 works great for zfs and has for many years now.
The LZ4 introduced in PostgreSQL 14 provides faster compression
4 projects | news.ycombinator.com | 8 Nov 2021
TokenCrypt – Compressor, Encryptor and Archiver
1 project | reddit.com/r/CKsTechNews | 25 Dec 2021
I just took a random screenshot and made it look prettier. [ I don't know if this counts as fanart ]
2 projects | reddit.com/r/PixelGun | 20 Dec 2021
Zstandard RFC 8878
2 projects | news.ycombinator.com | 19 Nov 2021
Slight upgrade lads
1 project | reddit.com/r/2hujerk | 11 Nov 2021