yjs-sqlite-test VS electric

I agree that full stack support is the missing pice that's need to make use explode. But I do think the current implementations will get there.

CRDTs are fairly unique in that you need them to be exposed very close to the front of your stack in order to capture user intent, but you also need support further back in your stack for merging, replication, and querying.

We have good front end support and there are multiple exciting projects building collaboration servers for them. What I think is missing is support in database, that's what I've been experimenting with (below). If you are building an offline enabled app, having the ability to generate diffs and merge in database enables easy multi document sync.

Also most general purpose CRDTs are a combination of JSON and XML like data structures, it's useful to be able to query the structures in your database. For example if you build a notes app that supports inline tags, if useful to be able to query and index those from within the XML like structure without having to dump the whole thing out at another layer of your stack.

Yjs Postgres: https://github.com/samwillis/yjs-pg-test

Yjs SQLite: https://github.com/samwillis/yjs-sqlite-test

(These are just early experiments, I'm working on a cleaner shared implementation with support for various SQLite bindings, and better querying)

Day job (contract): Massive overhaul of an antibody workbench for drug discovery.

Evenings: Building out a concept for a "YSQL" bringing CRDTs (Yjs) to SQLite and Postgres. CRDTs are great for real-time, but also awesome for offline for async collaboration. Using the same CRDT for both front end data structures and backend database merging seems to me to be a good combination. The plan is to build out some simple primitives first, then layer a SQLite (WASM in browser) <-> Postgres sync system on top for local copying and modification of datasets.

Proof of concepts:

https://github.com/samwillis/yjs-sqlite-test

https://github.com/samwillis/yjs-pg-test

I'm interested in this problem also!

I think there is a large overlap with projects that market/focus on offline-first experiences.

AFAIK this problem can be solved by:

1) Considering a client-side copy of the database that gets synced with the remote DB. This is an approach [PowerSync](https://www.powersync.com/) and [ElectricSql](https://electric-sql.com/) and [rxdb](https://rxdb.info/) take!

Yeah I agree with it being an exaggeration. They are certainly riding the admittedly dated perception that realtime is so hard it's only available to the Googles and Figmas. But there's now some amazing open source solutions available like Y.js and ElectricSQL[1]. The barrier has certainly come down.

[1] https://github.com/electric-sql/electric

https://github.com/electric-sql/electric :

> ElectricSQL is a local-first software platform that makes it easy to develop high-quality, modern apps with instant reactivity, realtime multi-user collaboration and conflict-free offline support.

> Local-first is a new development paradigm where your app code talks directly to an embedded local database and data syncs in the background via active-active database replication. Because the app code talks directly to a local database, apps feel instant. Because data syncs in the background via active-active replication it naturally supports multi-user collaboration and conflict-free offline

"SQLedge: Replicate Postgres to SQLite on the Edge" (2023)

Yup, this is correct.

We have had some requests / discussions around adding hooks to the sync service that will support custom logic on the write path (as per https://github.com/electric-sql/electric/discussions/565). This seems like a good idea but they don't exist yet.