pg_crdt VS yjs-sqlite-test

Cool, I'll check out your repos. Supabase also as Postgres CRDT (https://github.com/supabase/pg_crdt).

But, in general, I think searching in and across documents and database storage is going to a thorny problem even with existing, hyper-optimized CRDT algorithms. For example, if you just store your Yjs document as a binary blob in a Postgres column, Postgres becomes the bottleneck and all the fancy range-tree or b-tree optimizations are no longer that helpful on the server. Plus, it'd be great to have an document edit just be a tiny insert into the database that can be streamed to other clients rather than a big row update.

This all may because the focus of CRDTs seems to be rooted in a fully decentralized, P2P system but many developers want collaborative text editing in a more traditional client-server model.

I'm part of the Supabase Realtime team. Thanks for doing this! We've been wanting to do it internally ever since launching Broadcast. We've also been experimenting with what a very integrated CRDT solution looks like (e.g. pg_crdt).

This is an experimental extension for CRDTs, `pg_crdt`[0]. It supports Yjs/Yrs and Automerge.

The linked blog post describes how we're thinking about this extension in a Supabase context. Ideally this "Show HN" generates some discussion/interest, both here and in the github discussions [1].

I want to emphasise this part from the blog post[2]: "pg_crdt has not been released onto the Supabase platform (and it may never be). We’re considering many options for offline-sync/support and, while CRDTs will undoubtedly factor in, we’re not sure if this is the right approach."

[0] GitHub repo: https://github.com/supabase/pg_crdt

[1] Discussions: https://github.com/supabase/pg_crdt/discussions

[2] Blog post: https://supabase.com/blog/postgres-crdt

I agree that full stack support is the missing pice that's need to make use explode. But I do think the current implementations will get there.

CRDTs are fairly unique in that you need them to be exposed very close to the front of your stack in order to capture user intent, but you also need support further back in your stack for merging, replication, and querying.

We have good front end support and there are multiple exciting projects building collaboration servers for them. What I think is missing is support in database, that's what I've been experimenting with (below). If you are building an offline enabled app, having the ability to generate diffs and merge in database enables easy multi document sync.

Also most general purpose CRDTs are a combination of JSON and XML like data structures, it's useful to be able to query the structures in your database. For example if you build a notes app that supports inline tags, if useful to be able to query and index those from within the XML like structure without having to dump the whole thing out at another layer of your stack.

Yjs Postgres: https://github.com/samwillis/yjs-pg-test

Yjs SQLite: https://github.com/samwillis/yjs-sqlite-test

(These are just early experiments, I'm working on a cleaner shared implementation with support for various SQLite bindings, and better querying)

Day job (contract): Massive overhaul of an antibody workbench for drug discovery.

Evenings: Building out a concept for a "YSQL" bringing CRDTs (Yjs) to SQLite and Postgres. CRDTs are great for real-time, but also awesome for offline for async collaboration. Using the same CRDT for both front end data structures and backend database merging seems to me to be a good combination. The plan is to build out some simple primitives first, then layer a SQLite (WASM in browser) <-> Postgres sync system on top for local copying and modification of datasets.

Proof of concepts:

https://github.com/samwillis/yjs-sqlite-test

https://github.com/samwillis/yjs-pg-test