automerge-rs
peritext
Our great sponsors
automerge-rs | peritext | |
---|---|---|
12 | 20 | |
1,018 | 615 | |
- | 2.6% | |
9.8 | 0.0 | |
about 1 year ago | over 1 year ago | |
JavaScript | TypeScript | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
automerge-rs
-
Automerge 2.0
See also, Autosurgeon (with a 0.3.0 release today), which is a higher level API on top of Automerge for Rust:
I'm building a mobile app with a server backend, and I was looking for resources to build them in an offline-first way (since unlike on the browser, people expect to use apps offline, if they can, such as fitness or habit trackers).
I found the concept of conflict-free relational data types (CRDTS) interesting as it allows you to have fully offline experiences while also having a conflict-free syncing experience. I was looking for some good libraries and came across automerge [0] and yrs [1], but both had some rough APIs as they're primarily low-level Rust libraries that are wrapped by higher-level TypeScript APIs.
Autosurgeon wraps the low-level API of automerge to make it much more ergonomic, closer to the TypeScript experience, but in Rust of course. You can for example use `struct`s which autosurgeon will serialize and deserialize automatically, which is not present in base automerge, which focuses more on string keys and arbitrary values.
I am planning on using this together with Flutter and flutter_rust_bridge [2] in order to use this same Rust library everywhere. In this case, the server just becomes another (albeit more privileged) client.
[0] https://github.com/automerge/automerge-rs
-
Autosurgeon 0.3.0, use conflict-free replicated data types (CRDTs) to build offline-first apps with an easy-to-use API based on Automerge
I found the concept of conflict-free relational data types (CRDTS) interesting as it allows you to have fully offline experiences while also having a conflict-free syncing experience. I was looking for some good libraries and came across automerge and yrs, but both had some rough APIs as they're primarily low-level libraries that are wrapped by TypeScript APIs.
-
What do you recommend for conflict-free replicated data type (CRDT) support in Rust?
Yes, the plan is to use PostgreSQL. I had a discussion with one of the devs in this ticket about the strategy for this.
-
Some key-value storage engines in Rust
In any case, my current plan is to use Automerge for the data handling itself (so I can easily do collaboration), but that crate doesn't handle on-disk storage. For this I need another solution, and a K/V store is well suited for this task.
-
Official /r/rust "Who's Hiring" thread for job-seekers and job-offerers [Rust 1.66]
15 years working in software, Rust has been my favourite language for the last 2. Recently completed a contract to prototype a distributed Tailscale-inspired VPN built on Ink and Switch's CRDT project automerge-rs.
-
You might not need a CRDT (Conflict-free Replicated Data Type)
Complex topic. There's a very easy-to-use CRDT library for Rust (automerge), while there isn't much support for operational transforms (although Aper is new to me, I have to check it out).
-
Testing CRDTs in Rust, From Theory to Practice
I've been watching automerge-rs like a hawk, because they seem to be the only CRDT implementation where you don't need a Mathematics master to understand how to use it. They've been working on a rewrite for the last two years, hopefully they'll do a new release soon.
- Automerge: A JSON-like data structure (a CRDT) that can be modified concurrently
- Automerge: a new foundation for collaboration software [video]
-
Show HN: SyncedStore CRDT – build multiplayer collaborative apps for React / Vue
https://github.com/automerge/automerge-rs
By the way despite that particular repo (@localfirst/state) last being touched 6 months ago, Herb Caudill definitely seems still active in this space (I believe he's been working on other parts of this more recently -- e.g. ideas about authentication), and I think automerge development itself is quite active right now leading up to a 1.0 release which seems fairly imminent, for which a lot of fundamental work has been done, also coordinating with automerge-rs.
peritext
-
Cola: A text CRDT for real-time collaborative editing
This doesn’t appear to support rich text formatting ranges like bold, italic, etc - unless I’m missing something in the API. AFAIK Peritext is still the state of the art in rich text CRDT algorithms https://www.inkandswitch.com/peritext/
I’d love to see this build the rich text stuff from the Peritext algorithm.
-
The Cloud Is a Prison. Can the Local-First Software Movement Set Us Free?
The work Ink & Switch (unaffiliated) do has been an inspiration to my with regard to local-first and decentralized software: https://www.inkandswitch.com
They have a quasi-manifesto on local-first (https://www.inkandswitch.com/local-first/) and have published the best rich text CRDT around, Peritext: https://www.inkandswitch.com/peritext/
Lots of interesting work happening in this space.
-
Figma Is a File Editor
Take a look at https://automerge.org/ and the stack those folks are building. You're exactly right that it's a difficult balance (specifically the trick is proving commutativity for the domain-specific data of your application). But automerge (and then https://github.com/inkandswitch/peritext) show it's at least possible. Good stuff.
-
Ask HN: What is new in Algorithms / Data Structures these days?
Yes - The BFT problem only matters when you have Byzantine actors. But I think users deserve and expect the system to be reasonably well behaved and predictable in all situations. Anything publically writable, for example, needs BFT resilience. Or any video game.
As for the prosemirror problem, I assume you’re talking about weird merges from users putting markdown in a text crdt? You’re totally right - this is a problem. Text CRDTs treat documents as a simple sequence of characters. And that confuses a lot of structured formats. For example, if two users concurrently bold the same word, the system should see that users agree that it should be bolded. But if that “bold” intent is translated into “insert double asterisks here and here”, you end up with 4 asterisks before and after the text, and that confused markdown parsers. The problem is that a text crdt doesn’t understand markdown.
JSON editing has similar problems. I’ve heard of plenty of people over the years putting json text into a text crdt, only to find that when concurrent edits happen, the json grows parse errors. Eg if two users concurrently insert “a” and “b” into an empty list. The result is [“a””b”] which can’t be parsed.
The answer to both of these problems is to use CRDTs which understand the shape of your data structure. Eg, use a json OT/crdt system for json data (like sharedb or automerge). Likewise, if the user is editing rich text in prosemirror then you want a rich text crdt like peritext. Rich text CRDTs add the concept of annotations - so if two users bold overlapping regions of text, the crdt understands that the result should be that the entire region is bolded. And that can be translated back to markdown if you want.
The ink & switch people did a great write up of how this sort of crdt works here: https://www.inkandswitch.com/peritext/
- Edge cases in collaborative rich text editing (2021)
-
You might not need a CRDT
> I'm looking out for practical CRDT ideas that works well with richtext.
Have you seen Peritext from Ink & Switch? https://www.inkandswitch.com/peritext/ It's relatively new, but is a CRDT aimed at rich text!
-
CRDTs make multiplayer text editing part of Zed's DNA
To put it in a different perspective, plain text editing has well-solved CRDT patterns. But, semantic data-structures like rich-text or syntax trees is what's tricky and has unsolved challenges.
Peritext[1] is the only one that came close to solving rich-text, but even that one left out important aspect of rich-text editing like handling list & table operations as "work to be done later".
For people interested on why it's difficult to build CRDTs for richtext, here's a piece I wrote a year back: https://writer.zohopublic.com/writer/published/grcwy5c699d67...
Related HN discussion: https://news.ycombinator.com/item?id=29433896
- Peritext – A CRDT for Rich-Text Collaboration
-
Evan Wallace CRDT Algorithms
Anyone unsure of what a CRDT is, this is the perfect intro: https://www.inkandswitch.com/peritext/
The two most widely used CRDT implementations (combining JSON like general purpose types and rich text editing types) are:
- Automerge https://github.com/automerge/automerge
-
Is Svelte capable of a Google Docs & Sheets clone?
Svelte is, but that is your smallest problem. You want to look into CRDTs (conflict-free replicated data types) to offer true (offline) collaboration. A popular JS library to solve this complex problem is called [automerge](Conflict-free replicated data type). A rather recent development in that area specifically for text-based content is Peritext. Also check out this interactive tutorial about CRDTs.
What are some alternatives?
yjs - Shared data types for building collaborative software
automerge - A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically.
y-crdt - Rust port of Yjs
rust-libp2p - The Rust Implementation of the libp2p networking stack.
dokieli - :bulb: dokieli is a clientside editor for decentralised article publishing, annotations and social interactions
threlte - 3D framework for Svelte
SyncedStore - SyncedStore CRDT is an easy-to-use library for building live, collaborative applications that sync automatically.
rust-crdt - a collection of well-tested, serializable CRDTs for Rust
slate-yjs - Yjs binding for Slate