macos-cross-compiler VS unxip

Make is excellent if you use it properly to model your dependencies. This works really well for languages like C/C++, but I think Make really struggles with languages like Go, JavaScript, and Python or when your using a large combination of technologies.

I've found Earthly [0] to be the _perfect_ tool to replace Make. It's a familiar syntax (combination of Dockerfiles + Makefiles). Every target is run in an isolated Docker container, and each target can copy files from other targets. This allows Earthly to perform caching and parallelization for free, and in addition you get lots of safety with containerization. I've been using Earthly for a couple of years now and I love it.

Some things I've built with it:

* At work [1], we use it to build Docker images for E2E testing. This includes building a Go project, our mkdocs documentation, our Vue UI, and a ton of little scripts all over the place for generating documentation, release notes, dependency information (like the licenses of our deps), etc.

* I used it to create my macOS cross compiler project [2].

* A project for playing a collaborative game of Pokemon on Discord [3]

IMO Makefiles are great if you have a few small targets. If you're looking at more than >50 lines, if your project uses many languages, or you need to run targets in a Docker container, then Earthly is a great choice.

[0]: https://earthly.dev/

[1]: https://p3m.dev/

[2]: https://github.com/shepherdjerred/macos-cross-compiler

[3]: https://github.com/shepherdjerred/discord-plays-pokemon

It will depend heavily on the docker image you're trying to ship. For example with macos-cross-compiler[0] the resulting binary is over 2GB. With python:alpine[1] it's only 25MB.

Because image isn't copied whether the image is 2GB or 25MB the startup time will be nearly instantaneous for both.

The runtime adds 6-7MB of overhead although I expect that this can be reduced to less than 3MB with some work.

[0]: https://github.com/shepherdjerred/macos-cross-compiler

> We also need to notarize the .app and the .dmg. Interestingly, you can only notarize .pkg, .dmg, and .app files (in .zips) — command-line tools can only be notarized if they’re embedded in one of the listed containers.

I don't think this is true, actually. I've been notarizing command-line builds of unxip by shoving it in a zip and uploading it to notarytool for a while: https://github.com/saagarjha/unxip/blob/main/release.sh. (You will note, amusingly, that I am suffering from the same -parse-as-library bug as the author.) After this is done I just chuck the zip file and upload the binary directly to GitHub. If you download the file from your browser and chmod +x it, you can double click on it and it runs. Obviously you can skip the chmod +x step by sending people the zip (which preserves permissions inside of itself).

In theory I have all of those, but currently I have none, so it's manual work. Your best friend in diagnosing a kernel crash is a KDK. If you have one that matches your build, it will have symbols in it. With a little bit of math you can take the backtrace in the crash log and slide it appropriately to match the binary. Personally I use LLDB for this. Here's an example of what this looks like on an x86-64 kernel (Apple silicon has its own math but it's largely the same): https://github.com/saagarjha/unxip/issues/14#issuecomment-10.... The kernel is typically compiled with optimization, so there's a lot of inlining and code folding, but with function names, source files, and instruction offsets it's pretty trivial to match it to the code Apple publishes.

In this case I do not have a KDK for that build. In fact Apple has been unable to produce one for a couple of months, a inadequacy which I have repeatedly emphasized to them because of how critical they are for stuff like this. Supposedly they are working on it. Whatever; in lieu of that I got to figure out how good the tooling for analyzing kernels is these days, which was my real goal anyways.

For this crash log I downloaded the IPSW file for your build, 22A400. All of them get linked on The iPhone Wiki, e.g. https://www.theiphonewiki.com/wiki/Firmware/Mac/13.x. Once you unpack the IPSW (it's a zip file) there are compressed kernelcache files inside. Apple changed the format of these this year so most of the tooling breaks on it, but https://github.com/blacktop/ipsw was able to decompress them. Then I loaded it in to Binary Ninja, which apparently doesn't support them either but compiling this person's plugin (+166 submodules, and a LLVM & Boost build) gets it to work: https://github.com/skr0x1c0/binja_kc.

From there you can load up the faulting address from the crash log and see what the function looks like. In this case, a bunch of junk has been inlined into it but there's a really obvious and fairly unique string reference for "invalid knote %p detach, filter: %d". From there, you can compare it against the actual source code to see which one matches the "shape" of the function you're looking at. I happened to also pull up an older kernel which did have a KDK available and then compared its assembly to the new one to match it up to ptsd_kqops_detach. The disassembly of the crashing code is obviously doing a linked list walk so you can figure out exactly which line it is from that.

If I wasn't lazy I might also fire up a debugger to see why the function had walked off the end of the list but without KDKs things get pretty bad, not that they're very good to begin with. I don't have a m1n1 setup (I should probably do this at some point) and the things I do have, like remote debugging or the VM GDB stub, are not really worth suffering through for a Hacker News comment.

Use unxip https://github.com/saagarjha/unxip

> You may think that the numeric values (file_mode, file_size, file_mtime, ...) would be encoded in base 10, or maybe in hex, or using plain binary numbers ("base 256"). But no, they're actually encoded as octal strings (with a NUL terminator, or sometimes a space terminator). Tar is the only file format I know of which uses base 8 to encode numbers.

Tar’s “competitor”, cpio, does this as well (at least in one of the popular implementations). The Xcode XIP file, if you’re familiar with that particular format, is a couple layers of wrapping on top of a cpio archive, so deep inside my tool to expand these there’s a spot where I read these all out in a similar fashion: https://github.com/saagarjha/unxip/blob/5dcfe2c0f9578bc43070...

More generally, though, both tar and cpio doesn’t have an index because they don’t need them, they’re meant to be decompressed in a streaming fashion where this information isn’t necessary. It’s kind of inconvenient if you want to extract a single file, but for large archives you really want to compress files together and maintaining an index for this is both expensive and nontrivial, so I guess these formats are just “good enough”.

> This also applies to compression and decompression performed by Archive Utility: for example, if you download a copy of Xcode in xip format, decompressing that takes a long time as the code is constrained to the E cores, and there’s no way to change that.

This is actually not correct. If you decompress something with Archive Utility, you'll see that it does spend some time on all the cores, although it's definitely not using them equally. It's just that it doesn't spend very much time on them, because it does not parallelize tasks very well, so most of the time it will be effectively running on 1-2 threads with the other cores idle, which macOS will split over the efficiency cores and perhaps bring in one of the P-core clusters if it likes. For the very limited portions of the process that Archive Utility is actually computationally bottlenecked, AppleFSCompression will farm out the work to all the cores. When writing unxip (https://github.com/saagarjha/unxip) I found that you will actually lose to Archive Utility if you don't use all the cores at least part of the way.