libjpeg-turbo
libwebp
Our great sponsors
libjpeg-turbo | libwebp | |
---|---|---|
14 | 13 | |
3,553 | 1,896 | |
1.0% | 1.9% | |
8.4 | 8.7 | |
11 days ago | 7 days ago | |
C | C | |
GNU General Public License v3.0 or later | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
libjpeg-turbo
-
Why there may never be a libjpeg-turbo 3.1
> These projects deserve funding, if at least from giants like facebook & co.
Absolutely.
Looking at:
https://github.com/libjpeg-turbo/libjpeg-turbo/releases/tag/...
one wonders what possible harm could come from leaving image decompression buffer faults from maliciously crafted jpegs in popular browsers and software unattended.
While I think the move to safer code through Rust and other alternatives is a nice breath of fresh air, I doubt you can get these kinds of optimization without using unsafe code in Rust. These optimized implementations often require some kind of safety-bypassing memory modifications to work as efficiently ad they do.
There's a reason https://github.com/libjpeg-turbo/libjpeg-turbo/tree/main/sim... is filled with assembly files with conditional loading.
-
Learn x86-64 assembly by writing a GUI from scratch
Sure. You'll see it very often in codec implementations. From rav1e, a fast AV1 encoder mostly written in Rust: https://github.com/xiph/rav1e/tree/master/src/x86
Large portions of the algorithm have been translated into assembly for ARM and x86. Shaving even a couple percent off something like motion compensation search will add up to meaningful gains.
Or the current reference implementation of JPEG: https://github.com/libjpeg-turbo/libjpeg-turbo/tree/main/sim...
-
Announcing zune-jpeg: Rust's fastest JPEG decoder
zune-jpeg is 1.5x to 2x faster than jpeg-decoder and is on par with libjpeg-turbo.
-
JDK 21 - Image Performance Improvements
This is interesting from the standpoint of how new JVM features can be used to improve performance (what I presume the article's main purpose to have been), but the image processing improvement itself isn't head-turning. Also, we've found that libjpeg-turbo (https://libjpeg-turbo.org/) is ~5x (IIRC, can re-run my JMH benchmark if anyone wants me to) as fast for decoding JPEGs as ImageIO, so we wouldn't even benefit from this change in 21 much.
-
Convenient CPU feature detection and dispatch in the Magnum Engine
libjpeg-turbo: https://github.com/libjpeg-turbo/libjpeg-turbo/blob/main/simd/x86_64/jsimdcpu.asm
-
Looking for an open-source project to join part-time
There is no specific 'reference' that's required to match. However, you might be interested in libjpeg-turbo, which implements them in hand-written asm, as a comparison/bench target. My own opinion of this approach is that it should be unnecessary and for well-written code llvm beats 'hand-optimized' assembly. If some use of unsafe, unchecked indexing is necessary then so be it.
-
How do I create an image?
Different image formats are implemented by their own libraries, usually. libpng for PNG files, libjpeg-turbo is one of the ones for JPEG files, and so on.
-
The Future of Libjpeg-Turbo
https://news.ycombinator.com/item?id=25712301
This is based on https://github.com/mozilla/mozjpeg, which is a patched version of https://github.com/libjpeg-turbo/libjpeg-turbo
libwebp
-
Google assigns a CVE for libwebp and gives it a 10.0 score
The thing that concerns me most is looking at the fix it is very difficult to see why this fix is correct. It also appears as there is lots of code without explicit bounds checks. It makes me worried because while the logic may be safe this makes the logic very complex. I wonder what the cost would be to add an explicit, local bounds check at every array access. This would serve as a backup that is much easier to verify. I suspect the cost would be relatively small. Small enough that I personally would be happy to pay it.
https://github.com/webmproject/libwebp/commit/902bc919033134...
This is also a great reminded that fuzzing isn't a solution to memory unsafe languages and libraries. If anything the massive amount of bugs found via fuzzing should scare us as it is likely only scratching the surface of the vulnerabilities that still lie in the code, a couple too many branches away from being likely to be found by fuzzing.
-
The WebP 0day
Millions of people in the world are affected by this library. There is a call for libraries used by millions around the world to NOT use C. I love C. But this risk ratio is off the charts and they ought to not use C for such critical libraries.
I think this is the fix https://github.com/webmproject/libwebp/commit/dce8397fec159c...
"malloc fail"? :facepalm:
There's a follow-up fix, according to Debian[0]: https://github.com/webmproject/libwebp/commit/95ea5226c87044...
[0]: https://security-tracker.debian.org/tracker/CVE-2023-4863
This is the fix: https://github.com/webmproject/libwebp/commit/902bc919033134...
-
CVE-2023-4863: Heap buffer overflow in WebP (Chrome)
https://github.com/webmproject/libwebp/commit/95ea5226c87044...
This could mean Google optimized their fuzzers for libwebp after finding that bug and now they're finding more.
It's 2023, surely this is not yet another bug related to memory unsafety that could be avoided if we'd stop writing critical code that deals with extremely complex untrusted input (media codecs) in memory unsafe languages?
Yep, of course it is: https://github.com/webmproject/libwebp/commit/902bc919033134...
I guess libwebp could be excused as it was started when there were no alternatives, but even for new projects today we're still committing the same mistake[1][2][3].
[1] -- https://code.videolan.org/videolan/dav1d
[2] -- https://github.com/AOMediaCodec/libavif
[3] -- https://github.com/AOMediaCodec/libiamf
Yep. Keep writing these in C; surely nothing will go wrong.
The breakage [0] was introduced by the creator [1] of the project. If you want to audit 1674 commits over the past 12 years, it'd be easier to just audit the full project.
[0] https://github.com/webmproject/libwebp/commit/21735e06f7c1cb...
[1] https://github.com/webmproject/libwebp/commit/c3f41cb47e5f32...
The original commit in question: https://github.com/webmproject/libwebp/commit/f75dfbf23d1df1...
The commit that fixes this bug: https://github.com/webmproject/libwebp/commit/902bc919033134...
The original commit optimizes a Huffman decoder. The decoder uses a well-known optimization: it reads N bits in advance and determines how many bits have to be actually consumed and which symbol should be decoded, or, if it's an N-bit prefix of multiple symbols, which table should be consulted for remaining bits.
The old version did use lookup tables for short symbols, but longer symbols needed a graph traversal. The new version improved this by using an array of lookup tables. Each entry contains (nbits, value) where `nbits` is # bits to be consumed and `value` is normally a symbol, but if `nbits` exceeds N `value` is interpreted as a table index and `nbits` is reinterpreted as the longest code length in that subtree. So each subsequent table should have `2^(nbits - N)` entries (the root table is always fixed to 2^N entries).
The new version calculated the maximum number of entries based on the number of symbols (kTableSize). Of course, the Huffman tree comes from an untrusted source and you can easily imagine the case where `nbits` is very big. VP8 Lossless specifically allows up to 15 bits, so the largest possible table has 2^N + 2^15 entries when every single LUT is mapped to its own secondary table, and doing this doesn't need that many symbols (you only need 16-N symbols for each table). So if the Huffman tree was crafted in the way that maximizes the number of entries, it will overflow the allocation.
To be fair, I can see why this happened; the Huffman decoding step is one of the most computationally intensive part of many compression format and any small improvement matters. The Huffman decoder optimization described above is well known, but the longer code case is commonly considered less important to optimize because longer code should rarely appear in general. The original commit message refuted this, and was able to be merged.
- Convenient CPU feature detection and dispatch in the Magnum Engine
What are some alternatives?
ImageMagick - 🧙♂️ ImageMagick 7
orion - Usable, easy and safe pure-Rust crypto
bloom - The simplest way to de-Google your life and business: Inbox, Calendar, Files, Contacts & much more
virtualgl - Main VirtualGL repository
Rustup - The Rust toolchain installer
orion - Usable, easy and safe pure-Rust crypto [Moved to: https://github.com/orion-rs/orion]
Save-webP-as-extension - Firefox extension to overlay format and JPEG quality buttons on inline or stand-alone images for quickly saving a converted version of the image.
VisualMASM - Visual MASM - Assembly IDE for Microsoft MASM
BrowserBoxPro - :cyclone: BrowserBox is Web application virtualization via zero trust remote browser isolation and secure document gateway technology. Embed secure unrestricted webviews on any device in a regular webpage. Multiplayer embeddable browsers, open source! [Moved to: https://github.com/BrowserBox/BrowserBox]
layered-nlp - Rust natural language processing model with a focus on mapping back to source and "layerable" recognizers
go - The Go programming language
libavif - libavif - Library for encoding and decoding .avif files