JSON-joy CRDT benchmarks, 100x speed improvement over state-of-the-art

• json-joy

  2 626 9.9 TypeScript

  JSON CRDT, JSON CRDT Patch, JSON Patch+, JSON Predicate, CBOR, MessagePack, UBJSON, JSON Reactive RPC, JSON-RPC 2.0, JSON Pointer, JSON Expression, JSON Type

Love seeing CRDT innovation! I'm building something in the space, so excited to give this a try.

The death stroke for these types of projects seems to be lack of funding. This project is sponsored by nlnet[0] providing between 5k - 50k EU per year. Let's hope this gets additional resources.

As a note, it appears to use Elastic's 2.0 license preventing selling software that includes this library [1]

[0] https://nlnet.nl/project/JSON-Joy/

[1] https://github.com/streamich/json-joy/blob/master/LICENSE

• editing-traces

  1 38 6.8 JavaScript

  Real world text editing traces for benchmarking CRDT and Rope data structures

Hey! Author of diamond types and the (linked) editing traces repository here. Would it be possible to make & share an editing trace or two from your application? Even a single user editing trace would be super helpful - like a big list of which objects were replaced by what values, in order.

I really want json based CRDTs to be fast, but one of the problems we have optimising this stuff is that there aren’t a lot of real world data traces around to use as baselines for benchmarking. I don’t know which parts of automerge are slow, and without that knowledge I can’t make them fast. If we have some data from your application, most upcoming json based CRDTs will almost certainly work well for your use case.

If you’re up for it, flick me an email or just open a PR on https://github.com/josephg/editing-traces

• SurveyJS

  surveyjs.io sponsored

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• crdt-benchmarks

  8 402 0.0 JavaScript

  A collection of CRDT benchmarks

Author of Yjs here. I'm all for faster data structures. But only benchmarking one dimension looks quite fishy to me. A CRDT needs to be adequate at multiple dimensions. At least you should describe the tradeoffs in your article.

The time to insert characters is the least interesting property of a CRDT. It doesn't matter to the user whether a character is inserted within .1ms or .000000001ms. No human can type that fast.

It would be much more interesting to benchmark the time it takes to load a document containing X operations. Yjs & Yrs are pretty performant and conservative on memory here because they don't have to build an index (it's a tradeoff that we took consciously).

When benchmarking it is important to measure the right things and interpret the results somehow so that you can give recommendations when to use your algorithm / implementation. Some things can't be fast/low enough (e.g. time to load a document, time to apply updates, memory consumption, ..) other things only need to be adequate (e.g. time to insert a character into a document).

Unfortunately, a lot of academic papers set a bad trend of only measuring one dimension. Yeah, it's really easy to succeed in one dimension (e.g. memory or insertion-time) and it is very nice click-bait. But that doesn't make your CRDT a viable option in practice.

I maintain a set of benchmarks that tests multiple dimensions [1]. I'd love to receive a PR from you.

[1]: https://github.com/dmonad/crdt-benchmarks

Suggest a related project