lz4_flex
zstd
lz4_flex | zstd | |
---|---|---|
13 | 106 | |
411 | 22,407 | |
- | 1.4% | |
6.5 | 9.7 | |
21 days ago | 6 days ago | |
Rust | C | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
lz4_flex
-
Hetzner ARM cax11 vs Intel cx11 Benchmark
I run a benchmark based on my LZ4 implementation (de/compressor) between cax11 and cx11 since the instances cost the same. The code doesn't use any SIMD.
-
lz4_flex 0.11: Gainzzzzz Unleashed!: Performance Improvements Detailed in Blogpost (LZ4 De/compression)
By the way, the PR removing bounds checks from extend_from_within_overlapping is up: https://github.com/PSeitz/lz4_flex/pull/141
- lz4_flex (fast LZ4 de/compression) 0.10 released, now with legacy frame support ~ also 1Mio downloads ๐
-
ZAP: a VERY fast zip alternative, written in rust!
Zap uses LZ4 as it's primary compression algorithm, although I might add ZSTD as an option too. The LZ4 crate i'm using goes over the compression better than I can in the span of a comment. Check it out here.
-
lz4_flex 0.9 released
I wrote something here how hc could work: https://github.com/PSeitz/lz4_flex/issues/21
-
lz4_flex 0.8 released with support for frame format and major performance improvements
Yes, the updated benchmarks are here: https://github.com/pseitz/lz4_flex#results-v080-17-05-2021-safe-decode-and-safe-encode-off
- Lz4_flex โ fast LZ4 implementation in Rust
-
LZ4, an Extremely Fast Compression Algorithm
I ported the block format to Rust matching the C implementation in performance and ratio.
https://github.com/pseitz/lz4_flex
-
lz4_flex 0.7.2 reaches parity with cpp reference implementation on speed and ratio
Following this change count_same_bytes is unsound - it offsets the pointer by the value of candidate without any bounds checks, which may result in out-of-bounds access.
- lz4_flex 0.7 supports no_std (thanks @coolreader18), 32bit and is dependency-free
zstd
-
SQLite VFS for ZSTD seekable format
This VFS will read a sqlite file after it has been compressed using [zstd seekable format](https://github.com/facebook/zstd/blob/dev/contrib/seekable_f...). Built to support read-only databases for full-text search. Benchmarks are provided in README.
-
Chrome Feature: ZSTD Content-Encoding
Of course, you may get different results with another dataset.
gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]
zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]
NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.
NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.
NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases
For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10ร penalty on decompression.
xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
- Zstandard v1.5.6 โ Chrome Edition
-
Optimizating Rabin-Karp Hashing
Compression, synchronization and backup systems often use rolling hash to implement "content-defined chunking", an effective form of deduplication.
In optimized implementations, Rabin-Karp is likely to be the bottleneck. See for instance https://github.com/facebook/zstd/pull/2483 which replaces a Rabin-Karp variant by a >2x faster Gear-Hashing.
- Show HN: macOS-cross-compiler โ Compile binaries for macOS on Linux
-
Cyberpunk 2077 dev release
Get the data https://publicdistst.blob.core.windows.net/data/root.tar.zst magnet:?xt=urn:btih:84931cd80409ba6331f2fcfbe64ba64d4381aec5&dn=root.tar.zst How to extract https://github.com/facebook/zstd Linux (debian): `sudo apt install zstd` ``` tar -I 'zstd -d -T0' -xvf root.tar.zst ```
-
Honey, I shrunk the NPM package ยท Jamie Magee
I've done that experiment with zstd before.
https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...
Not sure about brotli though.
-
How in the world should we unpack archive.org zst files on Windows?
If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/2349
- Release Zstandard v1.5.5 ยท facebook/zstd
- ZSTD 1.5.5 is released with a corruption fix found at Google
What are some alternatives?
zfs - OpenZFS on Linux and FreeBSD
LZ4 - Extremely Fast Compression algorithm
Snappy - A fast compressor/decompressor
density - Superfast compression library
LZMA - (Unofficial) Git mirror of LZMA SDK releases
7-Zip-zstd - 7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
squashfs-tools-ng - A new set of tools and libraries for working with SquashFS images
ZLib - A massively spiffy yet delicately unobtrusive compression library.
zstd-rs - zstd-decoder in pure rust
brotli - Brotli compression format