ZLib VS libarchive

So the real issue here is that the lack of tree validation before the tree construction, I believe. I'm surprised that this check was not yet implemented (I actually checked libwebp to make sure that I was missing one). Given this blind spot, an automated test based on the domain knowledge is likely useless to catch this bug.

[1] https://github.com/madler/zlib/blob/master/examples/enough.c

In the source code of the Node.js opensource project, lib folder contains JavaScript code, mostly wrappers over C++ and function definitions. On the contrary, src folder contains C++ implementations of the functions, which pulls dependencies from the V8 project, the libuv project, the zlib project, the llhttp project, and many more - which are all placed at the deps folder.

A minimal viable Deflate decompressor is not exactly complex[1], although slower than mainline zlib.

[1] https://github.com/madler/zlib/tree/master/contrib/puff

ZLibrary not zlib for anyone worried about archiving algorithms. https://zlib.net/ and all the Linux repositories are safe (AFAIK).

>The following is my educated guess at what to do next and I am a week-end warrior programmer at best. If someone with more experience comments, I would listen to them over me< If you have no programming experience, this may be a lost cause. It looks like the save file has a header area followed by a list of "zlib" compressed chunks. zlib.net may have a program that can open the save file and let you see the raw information inside it. Something after the header is garbled and it is preventing Save-Editors from decompressing that chunk OR from parsing what is inside of it. If you are lucky, the garbled bit will jump out at you and you can repair it.

These appears to be the relevant changes:

2022-07-30: https://github.com/madler/zlib/commit/eff308af425b67093bab25...

2022-08-08: https://github.com/madler/zlib/commit/1eb7682f845ac9e9bf9ae3...

The second commit definitely fixed a null pointer dereference, I am not sure if the CVE is referencing something else that was fixed by the first commit.

It is actually possible to estimate the compression level without actually compressing everything again, because different levels use different strategies which can be identified in some cases. zlib in particular has three strategies [1] and preflate [2] leverages this to store the deflate stream reconstruction data without much bits.

[1] https://github.com/madler/zlib/blob/21767c6/deflate.c#L134-L...

[2] https://github.com/deus-libri/preflate/blob/master/preflate_...

It's still a heap of old school C spaghetti https://github.com/madler/zlib/commit/eff308af425b67093bab25...

I don't have a preview channel install handy to check, but apparently they're using libarchive so here's the full list assuming they expose everything it supports:

https://github.com/libarchive/libarchive/wiki/LibarchiveForm...

libarchive is an existing open source project. Replacing the older compression code with libarchive was probably not that complicated and they'll get bug fixes from that active open source project for free. I would not be surprised if the initial port to libarchive was done by one person in a single day.

And well, if you took a second to read the libarchive documentation (supported file formats) you'd find the following bullet point:

Read-only support (via libarchive). You still need the WinRAR app to create new RAR archives.

Be the change you want to see in this world!

IO speed is often a limiting factor, particularly with many small files. It might help to store the images in a zip file to avoid the overhead of opening and closing many files. It also improves throughput because in a zip file the small files will be stored contiguously while as regular files they'll each take up at least one disk block. When you store them in the zip file, disable compression because JPEGs are already compressed. Python has zip file access built-in, and in C you can use libzip or libarchive.

I am a noob and Rust is the first environment where I did cross compiling. I am also blown away by how easy and quick this works. However, with a new project it does not work anymore, because it involves a dynamic library (.so on Linux or .dll on Windows); specifically talking about libarchive with help of compress-tools.

libarchive ... twice