automerge-perf VS jdd

The automerge folks have a real-life editing history of a large document in their benchmarks: https://github.com/automerge/automerge-perf

This is an implementation problem with automerge. I wrote a blog post last year about CRDT performance. I re-ran the benchmarks a couple months ago. Automerge has improved a lot since then, but a simple benchmark test (automerge-perf[1]) still takes 200MB of RAM using automerge-rs. Yjs and Diamond types can run the same benchmark in 4mb / 2mb of ram respectively.

I've had a chat with some of the automerge people about it. They're working on it, and I've shared the techniques I'm using in diamond types (and all the code). Its just an implementation bottleneck.

[1] https://github.com/automerge/automerge-perf/

Without, try something like https://www.jsondiff.com/ - it should recognize the keys and point out differences visually.

> What's difficult is to ensure that the converged state is renderable as richtext. For example, is there a table cell that was inserted where a column was deleted?

Yes. This is one of the fundamental limitations of working at a textual level, which is sort of the local optimum that *nix ended up in. JSON particular gets suuuuper fucked up if you don't merge/rebase carefully. There's no real syntax for it to grab onto and diff doesn't understand the concept of indentation or commas, so it just turns into an ocean of line-swapping and incorrect block-swapping. Diff also does an excruciatingly poor job in the very common case when everyone is appending to the same area (let's say, the end of the file).

This is pretty much just an inherent weakness of textual matching, what you need is to work on trees of lexical token nodes, or some type of object structure stream like powershell.

In some cases patience-diff can help, it tries to generate big blocks of changed ranges, hopefully some of the hunks being syntactically well-formed commits. There is also JSON-diff which implements such a lexical-tree diff model for diff files, similar to the "jq" util. I think that's also viable for other lexable languages too.

https://github.com/zgrossbart/jdd