|8 days ago||4 days ago|
|GNU General Public License v3.0 or later||zlib License|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
How in the world should we unpack archive.org zst files on Windows?
2 projects | /r/Archiveteam | 24 May 2023
If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/23492 projects | /r/Archiveteam | 24 May 2023
ZSTD 1.5.5 is released with a corruption fix found at Google
3 projects | news.ycombinator.com | 4 Apr 2023
Float Compression 3: Filters
3 projects | news.ycombinator.com | 1 Feb 2023
Interesting to match with the observations from the practice of using ClickHouse for time series:
1. Reordering to SOA helps a lot - this is the whole point of column-oriented databases.
2. Specialized codecs like Gorilla, DoubleDelta, and FPC lose to simply using ZSTD compression in most cases, both in compression ratio and in performance.
3. Specialized time-series DBMS like InfluxDB or TimescaleDB lose to general-purpose relational OLAP DBMS like ClickHouse .
 https://arxiv.org/pdf/2204.09795.pdf "SciTS: A Benchmark for Time-Series Databases in Scientific Experiments and Industrial Internet of Things" (2022)
We're wasting money by only supporting gzip for raw DNA files
6 projects | news.ycombinator.com | 9 Jan 2023
zstd has a long range mode, which lets it find redundancies a gigabyte away. Try --long and --long=31 for very long range mode.
zstd has delta / patch mode, which creates a file that stores the "patch" to create a new file from an old (reference) file. See https://github.com/facebook/zstd/wiki/Zstandard-as-a-patchin...
See the man page: https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md
Decompressing the ZST files on Windows tips
2 projects | /r/pushshift | 20 Dec 2022
So I downloaded the Facebook tool https://github.com/facebook/zstd
They have a nice table on that page: https://github.com/facebook/zstd#benchmarks
Looking at that table, I think LZ4 is a winner. The compression ratio is not too far, compression speed is slightly faster, decompression speed is significantly faster, the code is much simpler so the compiled binary is smaller, and the project is unrelated to Facebook.
The checklist: Monitoring for Economy
3 projects | dev.to | 28 Nov 2022
In some cases, zstd can offer up to a 30% reduction in compressed storage as compared to other compression mechanisms. Learn more about zstd here.
Intel QuickAssist Technology Zstandard Plugin for Zstandard
10 projects | news.ycombinator.com | 16 Aug 2023
Introducing zune-inflate: The fastest Rust implementation of gzip/Zlib/DEFLATE
2 projects | /r/rust | 11 Feb 2023
It is much faster than miniz_oxide and all other safe-Rust implementations, and consistently beats even Zlib. The performance is roughly on par with zlib-ng - sometimes faster, sometimes slower. It is not (yet) as fast as the original libdeflate in C.
Zlib Critical Vulnerability
4 projects | news.ycombinator.com | 14 Oct 2022
Zlib-ng doesn't contain the same code, but it appears that their equivalent inflate() when used with their inflateGetHeader() implementation was affected by a similar problem: https://github.com/zlib-ng/zlib-ng/pull/1328
Also similarly, most client code will be unaffected because `state->head` will be NULL, because they (most client code) won't have used inflateGetHeader() at all.
Git’s database internals II: commit history queries
3 projects | news.ycombinator.com | 30 Aug 2022
I wonder if zlib-ng would make a difference, since it has a lot of optimizations for modern hardware.
Computing Adler32 Checksums at 41 GB/s
5 projects | news.ycombinator.com | 7 Aug 2022
zlib-ng also has adler32 implementations optimized for various architectures: https://github.com/zlib-ng/zlib-ng
Might be interesting to benchmark their implementation too to see how it compares.2 projects | news.ycombinator.com | 4 Aug 2022
Convenient CPU feature detection and dispatch in the Magnum Engine
9 projects | /r/cpp | 2 Aug 2022
games-emulation/dolphin-9999 is failing to build because devs switched to minizip-ng and zlib uses minizip. I'm not sure how to get it to build now, details in post.
2 projects | /r/Gentoo | 20 Jun 2022
(2) There are many packages that rely upon zlib and minizip and switching those underlying dependencies is easier said than done. We can't drop zlib completely and switch: "The idea of zlib-ng is not to replace zlib, but to co-exist as a drop-in replacement with a lower threshold for code change." - https://github.com/zlib-ng/zlib-ng
Re: Zlib memory corruption on deflate (i.e. compress)
4 projects | news.ycombinator.com | 28 Mar 2022
There are already active zlib forks (e.g. https://github.com/zlib-ng/zlib-ng), the problem is with having people move to them. It takes a lot of effort to move mindshare from the original version to a fork, there's some historical examples of it happening, but not a ton.
Zlib – a spiffy yet delicately unobtrusive compression library
8 projects | news.ycombinator.com | 24 Jan 2022
What are some alternatives?
LZ4 - Extremely Fast Compression algorithm
Snappy - A fast compressor/decompressor
LZMA - (Unofficial) Git mirror of LZMA SDK releases
7-Zip-zstd - 7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
ZLib - A massively spiffy yet delicately unobtrusive compression library.
brotli - Brotli compression format
haproxy - HAProxy Load Balancer's development branch (mirror of git.haproxy.org)
LZFSE - LZFSE compression library and command line tool
zlib - Cloudflare fork of zlib with massive performance improvements
zfs - OpenZFS on Linux and FreeBSD
LZHAM - Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++