openjpeg VS D2Evil

mkdir ~/install cd ~/install wget https://github.com/uclouvain/openjpeg/archive/master.zip unzip master.zip cd openjpeg-master/ mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release .. make sudo make install sudo make clean cd

Details here, if you like reading bug reports. It took the cooperation of developers at all four levels to fix this. I wrote a test project to reproduce the problem, the jpeg2k developer was able to reproduce the problem under valgrind (a tool for finding bad pointer references in C, seldom needed by Rust users), an OpenJPEG developer fixed the C code, and everybody in the chain updated versions. Thanks to everyone who had a part in this.

> The whole file is effectively a write-what-where primitive2. In addition to that, the Count Info Table is not bounds checked either...

File formats like that, with many offsets in the file, are troublesome. There used to be more formats like that. Microsoft Word .doc is the classic example.

OpenJPEG 2000 has a similar problem. I just hit that yesterday.[1] Valgrind is finding references to un-initialized data which affect control flow, and running the JPEG decoder on valid but truncated files (which is allowed) is causing bad memory reference crashes and errors.

New formats like this are rare. People have learned. A modern exception is Unreal Engine 5's Nanite has much offset data, and there may be an attack surface there for hostile game content. Nanite is a way to store a graphics mesh with both multiple levels of detail and common submeshes. It's a hierarchy of directed acyclic graphs, flattened into a linear file with offsets. And, sure enough, there are many crash reports. At least Unreal provides a validator for the format.

(If only C/C++ had slices in the language. Most of the things for which pointer arithmetic is used can be done with slices. Slices really are pointer arithmetic, with sanity.)

[1] https://github.com/uclouvain/openjpeg/issues/1459

JPEG 2000 decoder crashes Trying to read files at reduced resolution crashes with memory access errors. This is jpeg2k (Rust) calling jpeg2000-sys (Rust glue to C foreign function interfaces) calling OpenJPEG (all C). The problem can now be reproduced at the C level alone, under Valgrind. The developers of the top two levels have been very helpful, but the trouble is down in the rather painful C code at the bottom. The OpenJPEG maintainers are way behind on fixes, despite being funded.

It certainly was a risk in 2013, when the open source libraries were slow and riddled with bugs. Today, there are two stable, well maintained and fast open source libraries available:

https://github.com/uclouvain/openjpeg

I'm concerned it's a more fundamental issue. They're working with a closed-source format that is only supposed to be read/write by their closed-source tools. In the past, they've threatened legal action at someone just trying to document the formats. I imagine if someone digs at it more, they'll find that Live2D is trusting user input in general.

There's a bit more context to the story. The author and the company had a discussion over a reverse engineering claim as well : https://github.com/UlyssesWu/D2Evil/issues/1

I have to admit I'm somewhat impressed with the quality of the 2d puppetry I've seen in gaming recently (e.g. Marvel Snap, andy of the Gachas). It's a simpler technical skill than full 3d rigging that 2d illustrators seem to be able to be able to pick up fairly quickly.

The field this tool grew out of (mobile gaming) is such a grey ethical area, and these puppet animations are a fascinating form of pulp media.