libjpeg-turbo
zstd
Our great sponsors
libjpeg-turbo | zstd | |
---|---|---|
15 | 105 | |
3,582 | 22,356 | |
1.3% | 2.2% | |
8.4 | 9.7 | |
14 days ago | 9 days ago | |
C | C | |
GNU General Public License v3.0 or later | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
libjpeg-turbo
-
Jpegli: A New JPEG Coding Library
> all decoders will render the same pixels
Not true. Even just within libjpeg, there are three different IDCT implementations (jidctflt.c, jidctfst.c, jidctint.c) and they produce different pixels (it's a classic speed vs quality trade-off). It's spec-compliant to choose any of those.
A few years ago, in libjpeg-turbo, they changed the smoothing kernel used for decoding (incomplete) progressive JPEGs, from a 3x3 window to 5x5. This meant the decoder produced different pixels, but again, that's still valid:
https://github.com/libjpeg-turbo/libjpeg-turbo/commit/6d91e9...
-
My personal C coding style as of late 2023
Last vestiges of this fact AFAIK were libjpeg, which had a macro NEED_SHORT_EXTERNAL_NAMES that shortens all public identifiers to have unique 6-letter-long prefixes. Libjpeg-turbo nowadays has removed them though [1].
[1] https://github.com/libjpeg-turbo/libjpeg-turbo/commit/52ded8...
- Libjpeg-Turbo 3.0.0
-
Why there may never be a libjpeg-turbo 3.1
While I think the move to safer code through Rust and other alternatives is a nice breath of fresh air, I doubt you can get these kinds of optimization without using unsafe code in Rust. These optimized implementations often require some kind of safety-bypassing memory modifications to work as efficiently ad they do.
There's a reason https://github.com/libjpeg-turbo/libjpeg-turbo/tree/main/sim... is filled with assembly files with conditional loading.
-
Learn x86-64 assembly by writing a GUI from scratch
Sure. You'll see it very often in codec implementations. From rav1e, a fast AV1 encoder mostly written in Rust: https://github.com/xiph/rav1e/tree/master/src/x86
Large portions of the algorithm have been translated into assembly for ARM and x86. Shaving even a couple percent off something like motion compensation search will add up to meaningful gains.
Or the current reference implementation of JPEG: https://github.com/libjpeg-turbo/libjpeg-turbo/tree/main/sim...
-
Announcing zune-jpeg: Rust's fastest JPEG decoder
zune-jpeg is 1.5x to 2x faster than jpeg-decoder and is on par with libjpeg-turbo.
-
JDK 21 - Image Performance Improvements
This is interesting from the standpoint of how new JVM features can be used to improve performance (what I presume the article's main purpose to have been), but the image processing improvement itself isn't head-turning. Also, we've found that libjpeg-turbo (https://libjpeg-turbo.org/) is ~5x (IIRC, can re-run my JMH benchmark if anyone wants me to) as fast for decoding JPEGs as ImageIO, so we wouldn't even benefit from this change in 21 much.
-
Convenient CPU feature detection and dispatch in the Magnum Engine
libjpeg-turbo: https://github.com/libjpeg-turbo/libjpeg-turbo/blob/main/simd/x86_64/jsimdcpu.asm
-
Implementing SVE2 for Open Source Project
libjpeg-turbo
-
How to go about implementing file encoding [Question]
For all but the simplest formats (basically BMP), the difficulty of implementing encoding/decoding from scratch is significant - well beyond a beginner's ability, and challenging/time-consuming even for senior developers. So, libraries are used in practice - e.g. libpng and libjpeg-turbo.
zstd
-
Chrome Feature: ZSTD Content-Encoding
Of course, you may get different results with another dataset.
gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]
zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]
NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.
NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.
NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases
For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10× penalty on decompression.
xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
- Zstandard v1.5.6 – Chrome Edition
-
Optimizating Rabin-Karp Hashing
Compression, synchronization and backup systems often use rolling hash to implement "content-defined chunking", an effective form of deduplication.
In optimized implementations, Rabin-Karp is likely to be the bottleneck. See for instance https://github.com/facebook/zstd/pull/2483 which replaces a Rabin-Karp variant by a >2x faster Gear-Hashing.
- Show HN: macOS-cross-compiler – Compile binaries for macOS on Linux
-
Cyberpunk 2077 dev release
Get the data https://publicdistst.blob.core.windows.net/data/root.tar.zst magnet:?xt=urn:btih:84931cd80409ba6331f2fcfbe64ba64d4381aec5&dn=root.tar.zst How to extract https://github.com/facebook/zstd Linux (debian): `sudo apt install zstd` ``` tar -I 'zstd -d -T0' -xvf root.tar.zst ```
-
Honey, I shrunk the NPM package · Jamie Magee
I've done that experiment with zstd before.
https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...
Not sure about brotli though.
-
How in the world should we unpack archive.org zst files on Windows?
If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/2349
- Release Zstandard v1.5.5 · facebook/zstd
- ZSTD 1.5.5 is released with a corruption fix found at Google
-
zstd is used at Google
The story says : "ZSTD 1.5.5 is released with a corruption fix found at Google"
What are some alternatives?
ImageMagick - 🧙♂️ ImageMagick 7
LZ4 - Extremely Fast Compression algorithm
libwebp - Mirror only. Please do not send pull requests. See https://chromium.googlesource.com/webm/libwebp/+/HEAD/CONTRIBUTING.md.
Snappy - A fast compressor/decompressor
orion - Usable, easy and safe pure-Rust crypto
LZMA - (Unofficial) Git mirror of LZMA SDK releases
bloom - The simplest way to de-Google your life and business: Inbox, Calendar, Files, Contacts & much more
7-Zip-zstd - 7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
virtualgl - Main VirtualGL repository
ZLib - A massively spiffy yet delicately unobtrusive compression library.
Rustup - The Rust toolchain installer
brotli - Brotli compression format