libwebp VS zlib-ng

The thing that concerns me most is looking at the fix it is very difficult to see why this fix is correct. It also appears as there is lots of code without explicit bounds checks. It makes me worried because while the logic may be safe this makes the logic very complex. I wonder what the cost would be to add an explicit, local bounds check at every array access. This would serve as a backup that is much easier to verify. I suspect the cost would be relatively small. Small enough that I personally would be happy to pay it.

https://github.com/webmproject/libwebp/commit/902bc919033134...

This is also a great reminded that fuzzing isn't a solution to memory unsafe languages and libraries. If anything the massive amount of bugs found via fuzzing should scare us as it is likely only scratching the surface of the vulnerabilities that still lie in the code, a couple too many branches away from being likely to be found by fuzzing.

There's a follow-up fix, according to Debian[0]: https://github.com/webmproject/libwebp/commit/95ea5226c87044...

[0]: https://security-tracker.debian.org/tracker/CVE-2023-4863

The breakage [0] was introduced by the creator [1] of the project. If you want to audit 1674 commits over the past 12 years, it'd be easier to just audit the full project.

[0] https://github.com/webmproject/libwebp/commit/21735e06f7c1cb...

[1] https://github.com/webmproject/libwebp/commit/c3f41cb47e5f32...

If you like the command line, then you can use ffmpeg and ImageMagick, or use libwebp directly

The webp parser code is open source. Which means that even if Google decides to hide/obscure the code for webp, they'd legally not be allowed to prevent you from using older versions of the webp parser library. The only thing they could do is patent it, and then companies in the US (which has software patents, unfortunately) would have to pay royalties to decode it anyway; but here comes the next point

Please note that allowing for 2% bigger resulting file could mean huge speedup in these circumstances even with the same compression routines, seeing these benchmarks of zlib and zlib-ng for different compression levels:

https://github.com/zlib-ng/zlib-ng/discussions/871

IMO the fair comparison of the real speed improvement brought by a new program is only between the almost identical resulting compressed sizes.

It is much faster than miniz_oxide and all other safe-Rust implementations, and consistently beats even Zlib. The performance is roughly on par with zlib-ng - sometimes faster, sometimes slower. It is not (yet) as fast as the original libdeflate in C.

Zlib-ng doesn't contain the same code, but it appears that their equivalent inflate() when used with their inflateGetHeader() implementation was affected by a similar problem: https://github.com/zlib-ng/zlib-ng/pull/1328

Also similarly, most client code will be unaffected because `state->head` will be NULL, because they (most client code) won't have used inflateGetHeader() at all.

I wonder if zlib-ng would make a difference, since it has a lot of optimizations for modern hardware.

https://github.com/zlib-ng/zlib-ng/discussions/871

zlib-ng also has adler32 implementations optimized for various architectures: https://github.com/zlib-ng/zlib-ng

Might be interesting to benchmark their implementation too to see how it compares.

zlib-ng: https://github.com/zlib-ng/zlib-ng/blob/develop/functable.c

(2) There are many packages that rely upon zlib and minizip and switching those underlying dependencies is easier said than done. We can't drop zlib completely and switch: "The idea of zlib-ng is not to replace zlib, but to co-exist as a drop-in replacement with a lower threshold for code change." - https://github.com/zlib-ng/zlib-ng

There are already active zlib forks (e.g. https://github.com/zlib-ng/zlib-ng), the problem is with having people move to them. It takes a lot of effort to move mindshare from the original version to a fork, there's some historical examples of it happening, but not a ton.