Scala Streaming

Open-source Scala projects categorized as Streaming | Edit details

Top 10 Scala Streaming Projects

  • GitHub repo Akka

    Build highly concurrent, distributed, and resilient message-driven applications on the JVM

    Project mention: Best modern approaches to building composable distributed systems? | news.ycombinator.com | 2021-10-24
  • GitHub repo Scio

    A Scala API for Apache Beam and Google Cloud Dataflow.

    Project mention: ETL Pipelines with Airflow: The Good, the Bad and the Ugly | news.ycombinator.com | 2021-10-08

    If you prefer Scala, then you can try Scio: https://github.com/spotify/scio.

  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo Akka HTTP

    The Streaming-first HTTP server/module of Akka

    Project mention: Best Scala framework / libraries out there ? | reddit.com/r/scala | 2021-10-31

    Akka HTTP, Cats, Quill, ninny, Monix Observable, mill.

  • GitHub repo AkkaGRPC

    Akka gRPC

  • GitHub repo fs2-kafka

    Functional Kafka Streams for Scala

    Project mention: Need the advice to choose: FS2 vs ZIO-streams | reddit.com/r/scala | 2021-07-28

    If you only need statless transformations, I guess that fs2, zio-streams, and alpakka kafka are good solutions, and it's just a matter of personal preference. But zio and fs2 are definitely lightwheight when you compare it with akka streams. I see a lot of comments against ZIO streams that are not supported by the facts, since https://github.com/zio/zio-kafka has more contributors than https://github.com/fd4s/fs2-kafka. Here are some materials if you want to learn more about zio-kafka: https://www.youtube.com/watch?v=GECv1ONieLw https://scalac.io/blog/streaming-microservices-with-zio-and-kafka/

  • GitHub repo Iteratee

    Iteratees for Cats

  • GitHub repo Play Pagelets

    A module for the Play Framework to build highly modular applications

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo lila-ws

    Lichess' websocket server

    Project mention: In-game chat with pure socket programming | reddit.com/r/gamedev | 2021-06-09
  • GitHub repo fs2-es

    Event sourcing utilities for FS2

    Project mention: A naive question about modeling task | reddit.com/r/scala | 2021-08-23

    In pure FP terms, the system I'm aware of for this is fs2-es for the Typelevel ecosystem. Note that I haven't actually used it. I'm just aware that it exists.

  • GitHub repo s3-sqs-connector

    A library for reading data from Amzon S3 with optimised listing using Amazon SQS using Spark SQL Streaming ( or Structured streaming).

    Project mention: Upload to S3 -> AWS lambda with some Scala Spark code -> Process -> Write back to S3 | reddit.com/r/scala | 2021-05-27

    Are you planning on uploading and processing many files to S3? If so I would use something like Structured Streaming with the FileSource which can detect new files uploaded to S3 and process them in on a "standard" Spark cluster. You can then build a very easy to deploy and operate cluster on EKS/Kubernetes. I would check out: https://github.com/qubole/s3-sqs-connector once the number of files you upload start to get really large. Glue could also be used to achieve roughly the same thing and without the hassle of managing the EKS/K8s clusters.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-10-31.

Scala Streaming related posts

Index

What are some of the best open-source Streaming projects in Scala? This list will help you:

Project Stars
1 Akka 11,885
2 Scio 2,229
3 Akka HTTP 1,213
4 AkkaGRPC 386
5 fs2-kafka 208
6 Iteratee 187
7 Play Pagelets 76
8 lila-ws 73
9 fs2-es 73
10 s3-sqs-connector 11
Find remote jobs at our new job board 99remotejobs.com. There are 34 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com