Readyset: A MySQL and Postgres wire-compatible caching layer

• readyset

  24 3,882 9.8 Rust

  Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

  

In the Microsoft SQL Server space, several of these vendors have come and gone. My clients have been burned badly by 'em, so a few quick lessons learned:

Be aware that there are hundreds of open issues[0] and dozens of pull requests [1], some of which involve clients being unable to connect or not supporting all components of the SQL language. Just because your database supports something, doesn't mean your caching layer will.

It gets really ugly when a new version of your database comes out, with brand new features and language enhancements, and the caching layer doesn't support it. It may take months, or in some cases years, before the caching layer is feature-complete with the underlying database. If you want to use some of those language enhancements, then your app may have to maintain two connection strings - one for the caching layer, and one for direct database queries that the caching layer doesn't support.

Your support teams need to learn how to diagnose problems with the caching layer. For example, this issue [2] talks about the level of work involved with understanding why newly inserted data isn't showing up in selects.

I hope they succeed and deliver the concept, because it's one of the holy grails of databases.

[0]: https://github.com/readysettech/readyset/issues

• timely-dataflow

  11 3,145 7.2 Rust

  A modular implementation of timely dataflow in Rust

  

They have a bit about their technical foundation here[0].

Given that Readyset was co-founded by Jon Gjengset (but has apparently since departed the company), who authored the paper on Noria[1], I would assume that Readyset is the continuation of that research.

So it shares some roots with Materialize. They have a common conceptual ancestry in Naiad, where Materialize evolved out of timely-dataflow.

[0]: https://docs.readyset.io/concepts/streaming-dataflow

[1]: https://jon.thesquareplanet.com/papers/osdi18-noria.pdf

[2]: https://dl.acm.org/doi/10.1145/2517349.2522738

[3]: https://github.com/TimelyDataflow/timely-dataflow

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• hn-search

  1,618 524 2.9 TypeScript

  Hacker News Search

  

Wow, uh, Noria got listed a bunch of times looking back in search. https://hn.algolia.com/?q=noria

(It turns into Nokia results part way through page 2, but still a lot of submissions!)

Suggest a related project