zpaqlpy
fpnge
zpaqlpy | fpnge | |
---|---|---|
2 | 4 | |
19 | 79 | |
- | - | |
10.0 | 4.2 | |
over 1 year ago | about 2 months ago | |
Rust | C++ | |
GNU General Public License v3.0 only | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
zpaqlpy
-
Xz format considered inadequate for long-term archiving
ZPAQ is the name of the tool but ZPAQ is also the name of the container format that gets used. ZPAQ embeds the decompression algorithm in the archive. One could store zstd-compressed blocks in ZPAQ archives as soon as a zpaql decompressor exists (e.g., for brotli there is a slow one implemented in a python subset and compiled to zpaql https://github.com/pothos/zpaqlpy).
I don't know exactly whether other formats are better for seeking and streaming, but since the baseline is tar, ZPAQ (in the 2.0 spec) is already better as it supports deduplication and files can even be updated append-only, and the compression is not an afterthought wrapped around it but well integrated.
- QOI – The Quite OK Image Format
fpnge
-
png crate gets an ultrafast compression mode, up to 4x faster decompression
When the QOI format was first announced it wasn't clear that was even possible while keeping PNG format compatibility. But the fpng and fpnge C/C++ libraries showed it was, and today you can take advantage of those advances in a general purpose PNG library in Rust!
-
Hello, PNG
But most platforms these days have some form of CRC32 "acceleration". Adler32 is easy to compute so I'm even less concerned there.
I spent a bunch of time optimising the code in [fpnge](https://github.com/veluca93/fpnge), which is [often notably faster than fpng](https://github.com/nigeltao/qoir/blob/5671f584dcf84ddb71e28d...), yet checksum time is basically negligible.
Having said that, the double-checksum aspect of PNG does feel unnecessary.
- Computing Adler32 Checksums at 41 GB/s
- QOI – The Quite OK Image Format
What are some alternatives?
zfec - zfec -- an efficient, portable erasure coding tool
fpng - Super fast C++ .PNG writer/reader
zstd - Zstandard - Fast real-time compression algorithm
php-qoi - QOI image encoder and decoder written in pure PHP
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
canvas_ity - A tiny, single-header <canvas>-like 2D rasterizer for C++
restic - Fast, secure, efficient backup program
tinf - Tiny inflate library (inflate, gzip, zlib)
figlet-fonts - my collection of figlet / toilet ascii art fonts
dwarfs - A fast high compression read-only file system for Linux, Windows and macOS
zlib-ng - zlib replacement with optimizations for "next generation" systems.