hash-db VS pg_crdt

https://github.com/samsquire/hash-db

I need to combine the ideas in each of these projects into a cohesive solution.

I did some work on trying to implement the YATA algorithm, poorly.

From an SQL database implementation perspective, in my toy Python barebones SQL database that barely supports inner joins (https://github.com/samsquire/hash-db) I tested by testing on postgresql and seeing if my query with two joins produces the same results.

I ought to produce unit tests that prove that tuples from each join operation produces the correct dataset.

For a user perspective, I guess you could write some tooling that loads example data into a database and does an incremental join with each part of the join statement added.

I wrote a toy dynamodb, SQL, Cypher graph and document storage database engine in Python for the learning.

https://github.com/samsquire/hash-db

I am interested in database internals. Btrees come up with regard to designing database systems that are efficient to query on disk. Postgres uses them for its indexes. Radix trees are memory efficient tries which are useful for answering prefix queries. They're also called prefix trees. I use them to get a list of prefixes of a string. Useful for simple intellisense style forms or dynamodb style querying. I've also been studying LSM trees which are used in Leveldb and RocksDB.

I experiment with database technology in my experimental project hash-db https://github.com/samsquire/hash-db The code should be readable.

I need to change my search tree to be self balancing currently it grows to the left or right without balancing. I think I need to use tree rotation depending on which branch has the highest height.

Cool, I'll check out your repos. Supabase also as Postgres CRDT (https://github.com/supabase/pg_crdt).

But, in general, I think searching in and across documents and database storage is going to a thorny problem even with existing, hyper-optimized CRDT algorithms. For example, if you just store your Yjs document as a binary blob in a Postgres column, Postgres becomes the bottleneck and all the fancy range-tree or b-tree optimizations are no longer that helpful on the server. Plus, it'd be great to have an document edit just be a tiny insert into the database that can be streamed to other clients rather than a big row update.

This all may because the focus of CRDTs seems to be rooted in a fully decentralized, P2P system but many developers want collaborative text editing in a more traditional client-server model.

I'm part of the Supabase Realtime team. Thanks for doing this! We've been wanting to do it internally ever since launching Broadcast. We've also been experimenting with what a very integrated CRDT solution looks like (e.g. pg_crdt).

This is an experimental extension for CRDTs, `pg_crdt`[0]. It supports Yjs/Yrs and Automerge.

The linked blog post describes how we're thinking about this extension in a Supabase context. Ideally this "Show HN" generates some discussion/interest, both here and in the github discussions [1].

I want to emphasise this part from the blog post[2]: "pg_crdt has not been released onto the Supabase platform (and it may never be). We’re considering many options for offline-sync/support and, while CRDTs will undoubtedly factor in, we’re not sure if this is the right approach."

[0] GitHub repo: https://github.com/supabase/pg_crdt

[1] Discussions: https://github.com/supabase/pg_crdt/discussions

[2] Blog post: https://supabase.com/blog/postgres-crdt