An opinionated map of incremental and streaming systems (2018)

• differential-datalog

  22 1,334 0.0 Java

  DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner.

  

Checkout differential datalog. Looks super interesting: https://github.com/vmware/differential-datalog

But I've only played with souffle, which I am not sure where it lands in this taxonomy. I think it lands on the high temporal locality but compared to differential datalog, the biggest difference in my opinion is the souffle solves problems as a batch (i.e., all the inputs are known at command issue time) while differential datalog may receive inputs at runtime.

• flow

  2 1,477 3.4 Elixir

  Computational parallel flows on top of GenStage (by dashbitco)

  

Elixir has a few interesting abstractions for that: GenStage, Flow, Broadway.

https://github.com/dashbitco/flow

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• streaming-consistency

  3 19 1.8 Java

  Discontinued Demonstrations of (in)consistency in various streaming systems.

Spark structured streaming is in there under structured, high temporal locality.

It didn't make it into https://scattered-thoughts.net/writing/internal-consistency-... because it has severe limitations for low temporal locality operations:

> * As of Spark 2.4, you can use joins only when the query is in Append output mode. Other output modes are not yet supported.

• Pravega

  2 1,966 8.3 Java

  Pravega - Streaming as a new software defined storage primitive

  

Suggest a related project