sito VS notesutils

As of recent I've been weirdly fascinated with codecs, serialization protocols, file formats, and the like. It scratches that low-level hacking itch (I spent most of my professional programming in web dev space) without as much commitment to electronics (used to be super big into arduino but I find that's harder to pick up and put down).

I've started hacking on my own container format (yeah, I know, xkcd927), after finding it super frustrating to embed arbitrary time-synched data streams into mp4/matroska/ogg/etc. Also it bugs me how crusty, complicated, and arcane mp4 is, and at the same time, mastroska and ogg are weirdly opaque given how they are supposed to be open standards.

If anyone is curious, here's my container format I've been developing: https://github.com/xkortex/sito

I should write this stuff up, but I haven't.

I do have some brief notes on the "Notes.app" format here:

https://github.com/dunhamsteve/notesutils/blob/master/notes....

But I didn't discuss my methodology -- Generic decoding of protobuf, building up a schema as you go -- the tricky part there is that a byte array and a substructure look the same, so you have to try to decode it, and if successful, try that schema on the next example.

Here is another fun technique - scanning through a disassembly of an Apple framework looking for assembly patterns that match the protobuf compiler output (this was dependent on which language was targeted by protobuf):

https://gist.github.com/dunhamsteve/224e26a7f56689c33cea4f0f...

So you find the serializer / deserializer code and figure out what the original protbuf spec looked like.