-
eventually-consistent-mesh
an attempt to create an asynchronously replicated append only eventually consistent data protocol
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
https://github.com/samsquire/eventually-consistent-mesh/blob...
I wrote a Jepsen test that because it is not linerarizable, it fails.
My script abandons the C(onsistency) of CAP and maintains (A)vailability and (P)artition tolerance. It assumes the network shall heal and the replication shall resume. In PACELC theory in During normal operation it prefers Availability during Partitions and Else Latency in normal operations.
How do you build a system around a database where values can no longer be valid at any time? If I kick off a process in the database with a value that was only reported by one replica that was partitioned away from the majority of the network, what do I do?
My immediate thoughts are to create dependency relationships to values that are on the agreed path, data that is based on a failed branch is not valid.
https://github.com/samsquire/eventually-consistent-mesh
From an earlier blog post[1], the system for synchronizing file/document state lives independently for their database. As I understand it, their database is for metadata rather than actual document content.
> It’s worth noting that we only use multiplayer for syncing changes to Figma documents. We also sync changes to a lot of other data (comments, users, teams, projects, etc.) but that is stored in Postgres, not our multiplayer system, and is synced with clients using a completely separate system that won’t be discussed in this article. Although these two systems are similar, they have separate implementations because of different tradeoffs around certain properties such as performance, offline availability, and security.
This post[2] goes into more detail on how they spin up backend processes to serve as the source of truth while documents are open (we took heavy inspiration from it when building https://plane.dev, which aims to be an open-source implementation of that architecture.)
[1] https://www.figma.com/blog/how-figmas-multiplayer-technology...