differential-dataflow

An implementation of differential dataflow using timely dataflow on Rust. (by TimelyDataflow)

Differential-dataflow Alternatives

Similar projects and alternatives to differential-dataflow

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better differential-dataflow alternative or higher similarity.

differential-dataflow reviews and mentions

Posts with mentions or reviews of differential-dataflow. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-10-21.
  • We Built a Streaming SQL Engine
    3 projects | news.ycombinator.com | 21 Oct 2023
    Some recent solutions to this problem include Differential Dataflow and Materialize. It would be neat if postgres adopted something similar for live-updating materialized views.

    https://github.com/timelydataflow/differential-dataflow

    https://materialize.com/

  • Hydroflow: Dataflow Runtime in Rust
    5 projects | news.ycombinator.com | 7 Jun 2023
    I'm looking for this but can't find it, how does this project compare to differential dataflow?

    As a sibling commenter mentioned, it's built on timely dataflow (which is lower-level), but that already has differential dataflow[0] built on top of it by the same authors.

    How do they differ?

    [0]: https://github.com/TimelyDataflow/differential-dataflow

  • Using Rust to write a Data Pipeline. Thoughts. Musings.
    5 projects | /r/rust | 14 Jan 2023
  • PlanetScale Boost
    6 projects | news.ycombinator.com | 15 Nov 2022
  • Program Synthesis is Possible (2018)
    3 projects | news.ycombinator.com | 4 Sep 2022
  • Convex vs. Firebase
    7 projects | news.ycombinator.com | 21 Jun 2022
    hi! sujay from convex here. I remember reading about your "reverse query engine" when we were getting started last year and really liking that framing of the broadcast problem here.

    as james mentions, we entirely re-run the javascript function whenever we detect any of its inputs change. incrementality at this layer would be very difficult, since we're dealing with a general purpose programming language. also, since we fully sandbox and determinize these javascript "queries," the majority of the cost is in accessing the database.

    eventually, I'd like to explore "reverse query execution" on the boundary between javascript and the underlying data using an approach like differential dataflow [1]. the materialize folks [2] have made a lot of progress applying it for OLAP and readyset [3] is using similar techniques for OLTP.

    [1] https://github.com/TimelyDataflow/differential-dataflow

    [2] https://materialize.com/

    [3] https://readyset.io/

  • Announcing avalanche 0.1, a React- and Svelte-inspired GUI library
    6 projects | /r/rust | 30 Dec 2021
    differential dataflow which is used to power materialize db
  • Differential Datalog
    7 projects | news.ycombinator.com | 19 Mar 2021
    It's partially inspired by Linq, so the similarity you see is expected.

    It's not really arbitrary structures so much, though you're mostly free in what record type you use in a relation (structs and tagged enums are typical, though).

    The incremental part is that you can feed it changes to the input (additions/retractions of facts) and get changes to the outputs back with low latency (you can alternatively just use it to keep an index up-to-date, where you can quickly look up based on a key (like a materialized view in SQL)).

    This [0] section in the readme of the underlying incremental dataflow framework may help get the concept across, but feel free to follow up if you're still not seeing the incrementality.

    [0]: https://github.com/TimelyDataflow/differential-dataflow#an-e...

  • Dbt and Materialize
    3 projects | news.ycombinator.com | 1 Mar 2021
  • Materialized view questions
    1 project | /r/mit6824clojure | 28 Feb 2021
  • A note from our sponsor - WorkOS
    workos.com | 30 Apr 2024
    The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →

Stats

Basic differential-dataflow repo stats
14
2,473
8.3
4 days ago

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com