Our great sponsors
-
differential-datalog
DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner.
-
differential-dataflow
An implementation of differential dataflow using timely dataflow on Rust. (by frankmcsherry)
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Tutorial which I didn’t see linked in the README: https://github.com/vmware/differential-datalog/blob/master/d...
Lets you define queries over some data set declaratively, and instead of recomputing the query over the entire data set every time you want an updated answer, it uses Differential Dataflow <https://github.com/frankmcsherry/differential-dataflow> to efficiently(^1) calculate the new results by updating the results of the previous query execution in response to new updates to the data set.
^1: I'm not an expert on Differential Dataflow, so I don't know what "efficiently" means in this context, other than "should be faster than running the query from scratch."
If you like this kind of thing, google Self-Adjusting Computation.
There's a C++ library that implements it here: https://github.com/cmuparlay/psac
What is the difference with https://github.com/rust-lang/datafrog? It’s a Datalog engine written by Frank McSherry on top of differential dataflow, that’s used here also
If you click around a little, you end up on a blog post with this tidbit:
> This project got put together rather suddenly, in response to some work the Rust folks are doing[1] on their new and improved borrow checker.
I don't think I could tell you more than "Frank wrote it to help rust folks who were previously doing work with differential-dataflow directly."
1. https://github.com/rust-lang/polonius/pull/36#issuecomment-3...
We still have customers using DDlog, so we fix issues as they come up, but not working on any new features.
[1]: https://github.com/vmware/database-stream-processor
> I'm not sure if differential dataflow can provide this.
Yes, it can, but you will have to write the views yourself, in Rust.
Materialize (https://materialize.com) exists, though, and compiles SQL to differential-dataflow programs, in order to provide exactly what you're asking for.
(I work for Materialize)