Scala Stream Processing

Open-source Scala projects categorized as Stream Processing

Top 5 Scala Stream Processing Projects

  • Gearpump

    Lightweight real-time big data streaming engine over Akka

  • nussknacker

    Low-code tool for automating actions on real time data | Stream processing for the users.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • libretto

    Declarative concurrency and stream processing library for Scala

    Project mention: Synit – A Reactive Operating System | news.ycombinator.com | 2023-07-15

    I'm not sure this is actually easy to solve. The basic idea here seems to be that actors coordinate with the help of shared mutable state (the "dataspace"). At this point you're actually back to square one as synchronizing shared state is the core problem with distributed, concurrent systems…

    Given that, and the fact that this whole thingy is untyped makes me skeptical. Not even the std. ingredients like type-state, session types, behavioral types, or the like seem to get used. I also don't see linear types here, which go very well with session types / behavioral types, and can help making typing message passing easier.

    Also that this approach would be completely network transparent, as I see it, seems like a design flaw. One needs control over the various effects that can occur! Otherwise the points form "distributed systems future work" can't be solved, imho.

    And form the practical point of view: There is no Akka / Pekko based Scala implementation? I'm quite surprised, to be honest.

    Akka / Pekko is the actor runtime, and Scala is famous for it's eDSL features; besides of course its type-system, which makes thing like Libretto possible, which seems related here:

    https://github.com/TomasMikula/libretto

    Syndicate-lang looks OTOH more like the usual callback-hell.

    But like I've said, I don't get this whole thingy really. I've just clicked through some parts of the web-sites. Most likely I'm just overlooking the elephant in the room. So would be glad to learn more!

    (BTW: Very nice web-site design on all the projects. Especially nice typography. Really enjoyed this part while clicking through this stuff.)

  • fs2-kafka

    Kafka client for functional streams for scala (fs2) (by Spinoco)

  • ESPBench

    ESPBench - The Enterprise Stream Processing Benchmark

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-07-15.

Scala Stream Processing related posts

Index

What are some of the best open-source Stream Processing projects in Scala? This list will help you:

Project Stars
1 Gearpump 765
2 nussknacker 580
3 libretto 190
4 fs2-kafka 79
5 ESPBench 13
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com