rust-libp2p VS y-crdt

* (i.e. WebRTC: https://github.com/libp2p/rust-libp2p/tree/master/examples/browser-webrtc)

Substrate and Lighthouse use libp2p as a networking stack for communication between nodes. The libp2p framework is a versatile modular peer-to-peer networking stack. It provides a collections of abstractions, mechanisms, and protocols for facilitating communication in P2P systems. In particular, libp2p supports multiple transport mechanisms (TCP, QUIC, WebSocket, WebTransport, etc.), encryption schemes (TLS and Noise), and stream multiplexing. Higher-level protocols in libp2p are implemented on top of reliable, ordered, bidirectional binary streams, which are transparently encrypted and multiplexed by the framework.

It's a peer-to-peer "engine" with switchable components. Seems to run on different platforms (browsers, mobile, desktop, server).

At a glance, it looks pretty much like libp2p (https://libp2p.io/) but seems to integrate with libp2p as well (meaning you should be able to use Bifrost on one end, and libp2p on the other), so I'm guessing there is at least some fundamental difference, but I cannot spot it. Seems to use slightly different terminology compared to libp2p.

ComposeDB is a graph database created by 3BoxLabs, a company well-known in the Web3 ecosystem for their work on decentralized identifiers (DIDs) and their main product the Ceramic network. Ceramic is a network of nodes that store and share composable data streams on top of libp2p, the network stack that also powers IPFS.

Yes, there is already a Rust version (https://github.com/libp2p/rust-libp2p) that behaves well at this level but I think we can reach a higher level of performance on this point with Zig. Also, if you look at the long term roadmap of libp2p (https://github.com/libp2p/specs/blob/master/ROADMAP.md), the mobile devices and IoT integrations for example are part of the considerations.

libp2p

Definitely agree that would be valuable. In fact our multiplayer state synchronization aspect is largely implemented in TypeScript (there's a TS element to the stack as well), and is built on top of YJS – there is a Rust implementation of YJS already though, which would likely be a great start for anyone looking to build something similar purely in Rust: https://github.com/y-crdt/y-crdt

We are working on a plugin API, which will allow people to build functionality that leverages the multiplayer data model – but within the app rather than as standalone applications.

It sounds like you want a Conflict-Free Replicated Data Type, CRDT for short. There are some Rust libraries you can use, but y-crdt seems very feature-complete, a port of Yjs.

I've found reliably persistence on the backend irritating with yjs. Seems like the official path is to fork their example library and edit it. (The example is insufficient because, for example, it will silently eat data if the onchange webhook fails).

yrs initially looks tempting but it's unsound at it's core. (The plan is to port the API directly from JS, use unsafe to silence the borrow checker, then gradually fix specific instances of undefined behavior if they cause actual real world issues.[1] I don't this this is an approach that can work. That's a shame because a lot of impressive work has gone into yrs.)

[1]: https://github.com/y-crdt/y-crdt/issues/233

So exciting! Strangely enough, a couple of hours before this release, we just managed to wrap our heads around Yjs after playing with it on and off for a few weeks!

For anyone not up to date with the world of CRDTs, Seph Gentle's two blog posts have become legendary:

* https://josephg.com/blog/crdts-are-the-future/

* https://josephg.com/blog/crdts-go-brrr/

these are also worth checking out:

* https://github.com/y-crdt/y-crdt (rust implementation started by the creator of Yjs, Kevin Jahns)

* https://github.com/y-crdt/ypy (python bindings for the rust implementation)

* https://github.com/josephg/diamond-types (Seph Gentle's rust implementation of YATA, the algorith behind Yjs)

I found the concept of conflict-free relational data types (CRDTS) interesting as it allows you to have fully offline experiences while also having a conflict-free syncing experience. I was looking for some good libraries and came across automerge and yrs, but both had some rough APIs as they're primarily low-level libraries that are wrapped by TypeScript APIs.

Yrs (Yjs on Rust) maintainer here: we actually had some idea about building extension to Postgres ;) See: https://github.com/y-crdt/y-crdt/issues/220