hamt VS immer

This isn't a very good explanation. The wikipedia article isn't great either. I like this description:

https://github.com/mkirchner/hamt#persistent-hash-array-mapp...

The name does tell you quite a bit about what these are:

* Hash - rather than directly using the keys to navigate the structure, the keys are hashed, and the hashes are used for navigation. This turns potentially long, poorly-distributed keys into short, well-distributed keys. However, that does mean you have to compute a hash on every access, and have to deal with hash collisions. The mkirchner implementation above calls collisions "hash exhaustion", and deals with them using some generational hashing scheme. I think i'd fall back to collision lists until that was conclusively proven to be too slow.

* Trie - the tree is navigated by indexing nodes using chunks of the (hash of the) key, rather than comparing the keys in the node

* Array mapped - sparse nodes are compressed, using a bitmap to indicate which logical slots are occupied, and then only storing those. The bitmaps live in the parent node, rather than the node itself, i think? Presumably helps with fetching.

A HAMT contains a lot of small nodes. If every entry is a bitmap plus a pointer, then it's two words, and if we use five-bit chunks, then each node can be up to 32 entries, but i would imagine the majority are small, so a typical node might be 64 bytes. I worry that doing a malloc for each one would end up with a lot of overhead. Are HAMTs often implemented with some more custom memory management? Can you allocate a big block and then carve it up?

Could you do a slightly relaxed HAMT where nodes are not always fully compact, but sized to the smallest suitable power of two entries? That might let you use some sort of buddy allocation scheme. It would also let you insert and delete without having to reallocate the node. Although i suppose you can already do that by mapping a few empty slots.

I've been working on an editor (not text) in C++ and pretty early got into undo/redo. I went down the route of doIt/undoIt for commands but that quickly got old. There was both the extra work needed to implement undo separately for every operation, but also the nagging feeling that the undo operation for some operation wasn't implemented correctly.

In the end, I switched to representing the entire document state using persistent data structures (using the immer library). This vastly simplified things and implementing undo/redo becomes absolutely trivial when using persistent data structures. It's probably not something that is suitable for all domains, but worth checking out.

https://github.com/arximboldi/immer

How does this compare to https://github.com/arximboldi/immer (other than the C/C++ difference)?

Also, it's my understanding that, in practice, persistent data structures require a garbage collector in order to handle deallocation when used in a general-purpose way. How does your implementation handle that?

You might be interested in ewig and immer by Juan Pedro Bolivar Puente:

https://github.com/arximboldi/ewig

https://github.com/arximboldi/immer

See the author instantly opening a ~1GB text file with async loading, paging through, copying/pasting, and undoing/redoing in their prototype “ewig” text editor about 27 minutes into their talk here:

https://m.youtube.com/watch?v=sPhpelUfu8Q

It’s backed by a “vector of vectors” data structure called a relaxed radix balanced tree:

https://infoscience.epfl.ch/record/169879/files/RMTrees.pdf

That original paper has seen lots of attention and attempts at performance improvements, such as:

https://hypirion.com/musings/thesis

https://github.com/hyPiRion/c-rrb

You’re absolutely right, however people have been putting in the “extra efforts” required for efficiency. Check out immer if you’re interested.

The C++ immer library: https://github.com/arximboldi/immer

For C++ check this one out - https://github.com/arximboldi/immer

Yeah I agree that I should link that wikipedia page in the docs, I'll do that as soon as I get a chance. immer (https://github.com/arximboldi/immer) also links that page in its docs, for the exact same reason I'm sure. Interestingly, there is a lot of overlap between persistent data structures in the functional programming sense and persistent data structures in the persisted-to-disk sense because persistent data structures in the FP sense are one of the best ways to guarantee atomic updates and safe failure recovery in a persisted-to-disk system! Btrfs and ZFS, as well as many databases, are at their core basically just copy-on-write B+ trees.

I’ve never tried it, but I love the idea of implementing editor scene state using immutable data structures like https://github.com/arximboldi/immer With that, every edit would append a new node to a list of scene states. Undo/redo becomes iterating your view of the scene up and down through that list. Can’t screw up an undo function if there’s never any work to do :P

I have! I think it's related to the C++ immer library which I used several years ago in Vortex. It's kinda like the previous generation of ValueScript. 🍻