hamt VS asami

This isn't a very good explanation. The wikipedia article isn't great either. I like this description:

https://github.com/mkirchner/hamt#persistent-hash-array-mapp...

The name does tell you quite a bit about what these are:

* Hash - rather than directly using the keys to navigate the structure, the keys are hashed, and the hashes are used for navigation. This turns potentially long, poorly-distributed keys into short, well-distributed keys. However, that does mean you have to compute a hash on every access, and have to deal with hash collisions. The mkirchner implementation above calls collisions "hash exhaustion", and deals with them using some generational hashing scheme. I think i'd fall back to collision lists until that was conclusively proven to be too slow.

* Trie - the tree is navigated by indexing nodes using chunks of the (hash of the) key, rather than comparing the keys in the node

* Array mapped - sparse nodes are compressed, using a bitmap to indicate which logical slots are occupied, and then only storing those. The bitmaps live in the parent node, rather than the node itself, i think? Presumably helps with fetching.

A HAMT contains a lot of small nodes. If every entry is a bitmap plus a pointer, then it's two words, and if we use five-bit chunks, then each node can be up to 32 entries, but i would imagine the majority are small, so a typical node might be 64 bytes. I worry that doing a malloc for each one would end up with a lot of overhead. Are HAMTs often implemented with some more custom memory management? Can you allocate a big block and then carve it up?

Could you do a slightly relaxed HAMT where nodes are not always fully compact, but sized to the smallest suitable power of two entries? That might let you use some sort of buddy allocation scheme. It would also let you insert and delete without having to reallocate the node. Although i suppose you can already do that by mapping a few empty slots.

Unlike some other commenters, I agree that graph models are usually a better fit for most data than relational models. There's been some interesting work in recent years developing this idea: in the Clojure world there's Datomic, XTDB, and a host of competitors, all of which build on work from Semantic Web/SPARQL/triplestores and logic programming. Some are even intended to be used as primary datastores: they support some amount of schema and constraints, have well-defined consistency and ACID guarantees, etc. This makes them unlike graph databases like Neo4J and others, which fill an architectural role more like Elasticsearch as a read-optimization tool. Here's an interesting talk making a case for triple-based databases.

Despite the documentation that exists, I've heard many people who have been confused about how to query Datomic, Datascript, Asami, or other graph databases. So I've made an attempt at explaining it https://github.com/threatgrid/asami/wiki/Introduction

The first Graph implementation for Asami was a simple in-memory data structure, described in my ClojureD talk. The code for this appears in asami.index. This file started much smaller (as referenced above), but has since expanded with the needs extended functionality, such as transactions, and transitive closure operations.