zpaqlpy
fpng
zpaqlpy | fpng | |
---|---|---|
2 | 10 | |
19 | 830 | |
- | - | |
10.0 | 4.1 | |
over 1 year ago | 5 months ago | |
Rust | C | |
GNU General Public License v3.0 only | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
zpaqlpy
-
Xz format considered inadequate for long-term archiving
ZPAQ is the name of the tool but ZPAQ is also the name of the container format that gets used. ZPAQ embeds the decompression algorithm in the archive. One could store zstd-compressed blocks in ZPAQ archives as soon as a zpaql decompressor exists (e.g., for brotli there is a slow one implemented in a python subset and compiled to zpaql https://github.com/pothos/zpaqlpy).
I don't know exactly whether other formats are better for seeking and streaming, but since the baseline is tar, ZPAQ (in the 2.0 spec) is already better as it supports deduplication and files can even be updated append-only, and the compression is not an afterthought wrapped around it but well integrated.
- QOI – The Quite OK Image Format
fpng
-
png crate gets an ultrafast compression mode, up to 4x faster decompression
When the QOI format was first announced it wasn't clear that was even possible while keeping PNG format compatibility. But the fpng and fpnge C/C++ libraries showed it was, and today you can take advantage of those advances in a general purpose PNG library in Rust!
-
Hello, PNG
CRC is a table and 5 lines of code. That's trivial.
>zlib is 23k lines
It's not needed to make a PNG reader/writer. zlib is massive overkill for only making a PNG reader or writer. Here's a tiny deflate/inflate code [2] under 1k lines (and could be much smaller if needed).
stb[0] has single headers of ~7k lines total including all of the formats PNG, JPG, BMP,. PSD, GIF, HDR, and PIC. Here's [1] a 3k lines single file PNG version with tons if #ifdefs for all sorts of platforms. Removing those and I'd not be surprised if you could not do it in ~1k lines (which I'd consider quite simple compared to most of todays' media formats).
>Of course they're not common formats so you're stuck with complex formats like PNG
BMP is super common and easy to use anywhere.
I use flat image files all the time for quick and dirty stuff. They quickly saturate disk speeds and networking speeds (say recording a few decent speed cameras), and I've found PNG compression to alleviate those saturate CPU speeds (some libs are super slow, some are vastly faster). I've many times made custom compression formats to balance these for high performance tools when neither things like BMPs or things like PNG would suffice.
[0] https://github.com/nothings/stb
[1] https://github.com/richgel999/fpng/blob/main/src/fpng.cpp
[2] https://github.com/jibsen/tinf/tree/master/src
- Quite OK Image is now my favorite asset format
-
Computing Adler32 Checksums at 41 GB/s
This was actually considered, and other libraries do ignore checksums, or at least have options to:
https://github.com/richgel999/fpng/issues/9
-
QOI – The Quite OK Image Format
In the other direction, you can target a subset of PNG to get less optimized images but with QOI-like encode and decode speed: https://github.com/richgel999/fpng
-
ReShade 5.0 Released!
On specific operations like screenshots for example the new version is WAY faster though. We were using stb_image_write.h but switched to fpng which promised 12-19x faster compression at smaller sizes. That and the fact that screenshot saving have now been given it's own thread so it now longer causes a small stutter like when it was on the main thread, means that screenshot are now near instantaneous.
-
QOI – The “Quite OK Image Format” for fast, lossless image compression
I think QOI inspired the creation of https://github.com/richgel999/fpng which creates standard PNGs and compares itself directly to QOI.
-
Losslessly compresses RGB and RGBA images to a similar size of PNG, while offering a 20x-50x speedup in compression and 3x-4x speedup in decompression
BTW, today I found this fpng-fast PNG writer. There is a comparison with QOI in the readme.
What are some alternatives?
zfec - zfec -- an efficient, portable erasure coding tool
qoi - The “Quite OK Image Format” for fast, lossless image compression
zstd - Zstandard - Fast real-time compression algorithm
fpnge - Demo of a fast PNG encoder.
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
libdeflate - Heavily optimized library for DEFLATE/zlib/gzip compression and decompression
restic - Fast, secure, efficient backup program
oss-nvjpg - Hardware-accelerated JPEG decoding on the Nvidia Tegra X1
PNG-spec - Maintenance of the PNG specification
dwarfs - A fast high compression read-only file system for Linux, Windows and macOS
php-qoi - QOI image encoder and decoder written in pure PHP