You might not need a CRDT

• aper

  8 292 0.0 JavaScript

  A Rust data structure library built on state machines.

  

It's also super refreshing to see the work on Aper [1] [2] (a Rust library implementing state machine synchronization across a trusted network). Looking forward next series of articles here!

[1]: https://aper.dev/

[2]: https://github.com/drifting-in-space/aper

• plane

  23 1,590 9.4 Rust

  A distributed system for running WebSocket services at scale. (by drifting-in-space)

  

> I wonder what web development could learn from online game development when it comes to "multiplayer" apps - I mean it's right there in the name.

100%. The architecture we used for https://plane.dev takes a lot of inspiration from how game servers work, but using HTTP/WebSocket instead of UDP.

A lot of state sharing techniques that people are now using on the web (like cursor position smoothing and optimistic updates) have decades of precedent in multiplayer games.

We’ve also seen more apps start to use an approach to pixel streaming which more closely resembles stadia than traditional VNC. https://digest.browsertech.com/archive/browsertech-digest-wo...

• SurveyJS

  surveyjs.io featured

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• pigeon

  4 54 0.0 JavaScript

  Diff, patch, merge, and synchronize JSON documents with an Automerge-compatible interface (by frameable)

  

We have used Automerge a bunch, but found that there is a threshold where beyond a given document size, performance gets exponentially worse, until even trivial updates take many seconds' worth of CPU. That is often how it works when the document end state is exclusively the sum of all the edits that have ever happened.

Our answer was to reimplement the Automerge API with different mechanics underneath that allows for a "snapshots + recent changes" paradigm, instead of "the doc is the sum of all changes". That way performance doesn't have to degrade over time as changes accumulate.

Project is here: https://github.com/frameable/pigeon, with some benchmarks: https://github.com/frameable/pigeon/wiki/Benchmarks in the wiki...

• peritext

  20 615 0.0 TypeScript

  A CRDT for asynchronous rich-text collaboration, where authors can work independently and then merge their changes.

  

> I'm looking out for practical CRDT ideas that works well with richtext.

Have you seen Peritext from Ink & Switch? https://www.inkandswitch.com/peritext/ It's relatively new, but is a CRDT aimed at rich text!

• statebox_riak

  1 48 10.0 Erlang

  Convenience library that makes it easier to use statebox with riak, extracted from best practices in our production code at Mochi Media.

  

This is a cool approach. It reminds me of statebox by mochimedia: https://github.com/mochi/statebox_riak.

If I'm understanding correctly, it requires the mutations to be deterministic in order for the nodes to converge.

Replicache (replicache.dev - my thing) takes a similar approach except it does not requires the mutations to be deterministic, which is very useful because it enables, e.g., authenticated operations on the server.

Both the idea here and Replicache's approach are closely related to game networking. If you are interested in these ideas, a really excellent set of content is: https://www.gabrielgambetta.com/client-server-game-architect....

• automerge-perf

  2 35 3.2 JavaScript

  Performance tests for Automerge

  

This is an implementation problem with automerge. I wrote a blog post last year about CRDT performance. I re-ran the benchmarks a couple months ago. Automerge has improved a lot since then, but a simple benchmark test (automerge-perf[1]) still takes 200MB of RAM using automerge-rs. Yjs and Diamond types can run the same benchmark in 4mb / 2mb of ram respectively.

I've had a chat with some of the automerge people about it. They're working on it, and I've shared the techniques I'm using in diamond types (and all the code). Its just an implementation bottleneck.

[1] https://github.com/automerge/automerge-perf/

• jdd

  2 970 6.9 JavaScript

  A semantic JSON compare tool

> What's difficult is to ensure that the converged state is renderable as richtext. For example, is there a table cell that was inserted where a column was deleted?

Yes. This is one of the fundamental limitations of working at a textual level, which is sort of the local optimum that *nix ended up in. JSON particular gets suuuuper fucked up if you don't merge/rebase carefully. There's no real syntax for it to grab onto and diff doesn't understand the concept of indentation or commas, so it just turns into an ocean of line-swapping and incorrect block-swapping. Diff also does an excruciatingly poor job in the very common case when everyone is appending to the same area (let's say, the end of the file).

This is pretty much just an inherent weakness of textual matching, what you need is to work on trees of lexical token nodes, or some type of object structure stream like powershell.

In some cases patience-diff can help, it tries to generate big blocks of changed ranges, hopefully some of the hunks being syntactically well-formed commits. There is also JSON-diff which implements such a lexical-tree diff model for diff files, similar to the "jq" util. I think that's also viable for other lexable languages too.

https://github.com/zgrossbart/jdd

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• diamond-types

  15 1,431 9.2 Rust

  The world's fastest CRDT. WIP.

I'm working on a CRDT to solve this problem too[1]. How do you plan on implementing collaborative text editing on top of your event-reordering system? Off the top of my head I can't think of a way to implement text on your proposed system which would be performant and simple.

[1] https://github.com/josephg/diamond-types

Suggest a related project