Scala Scala

Open-source Scala projects categorized as Scala | Edit details

Top 23 Scala Scala Projects

  • GitHub repo Apache Spark

    Apache Spark - A unified analytics engine for large-scale data processing

    Project mention: How to use Spark and Pandas to prepare big data | dev.to | 2021-09-21

    Apache Spark is one of the most actively developed open-source projects in big data. The following code examples require that you have Spark set up and can execute Python code using the PySpark library. The examples also require that you have your data in Amazon S3 (Simple Storage Service). All this is set up on AWS EMR (Elastic MapReduce).

  • GitHub repo scala

    Scala 2 compiler and standard library. For bugs, see scala/bug

    Project mention: Scalqa: Boost data processing speed with heavy ops | reddit.com/r/scala | 2021-08-12

    I believe that's the older bit-mapped trie structure, which was replaced with Radix Balanced Finger Trees. Here's the PR where they did it.

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo Play

    Play Framework

    Project mention: Akka became the de-facto solution for Scala web development? | reddit.com/r/scala | 2021-09-18

    Play, blergh for some, not blergh for others ;)

  • GitHub repo kafka-manager

    CMAK is a tool for managing Apache Kafka clusters

    Project mention: We tried to make monitoring tool for Kafka | reddit.com/r/apachekafka | 2021-07-22
  • GitHub repo Lila

    ♞ lichess.org: the forever free, adless and open source chess server ♞

    Project mention: How to export links saved in workona to chrome bookmarks | reddit.com/r/chrome | 2021-09-22
  • GitHub repo Gitbucket

    A Git platform powered by Scala with easy installation, high extensibility & GitHub API compatibility

    Project mention: How to build a search engine with Ruby on Rails | news.ycombinator.com | 2021-09-16

    > Rails doesn't scale? Github's the largest code repository site in the world.

    You know, i think i understand both of the viewpoints here. Personally, i'd say that Rails doesn't scale as well as i'd expect it to. You can definitely build scalable systems in it, though you'll end up throwing a whole bunch of hardware resources, when compared to certain other languages and technology stacks, to serve similar load.

    For example, right now i self-host a GitLab (https://about.gitlab.com/) instance for managing my code repositories, CI builds and so on. Even with just me using it (alongside some automated processes), it routinely eats up close to 4 GB of RAM, which in my case is an entire VPSes worth and costs me about 60 Euros a year with Time4VPS (affiliate link, if you'd like to check it out: https://www.time4vps.com/?affid=5294) but would cost me way more in AWS, GCP etc. One could argue that that's not too expensive, but not everyone earns a lot of money and running 10-20 VPSes does eventually build up, since i can't afford colocation and my residential homelab setup with a WireGuard tunnel to bypass ISP NAT with a proxy VPS is pretty slow, even if i can afford more storage, RAM and CPU power that way.

    Compare that situation to projects like Gogs (https://gogs.io/), Gitea (https://gitea.com/), GitBucket (https://gitbucket.github.io/) and sourcehut (https://sourcehut.org/) - i'd argue that all of them on average use less CPU resources and memory for accomplishing similar tasks. For example, have a look here: https://forgeperf.org/

    However, we cannot ignore the fact that using Ruby might have been exactly what allowed for quickly creating the functionality of GitLab and many other platforms and tools out there, GitHub included, so the choice between usable software and innovation in the near future and performant software possibly years from now is a tricky one.

    There are probably good arguments for both, but noone can declare either to be better. Personally, i don't mind using Ruby, Python or even PHP when it makes sense and i don't need to worry about scalability from day 0.

  • GitHub repo Finagle

    A fault tolerant, protocol-agnostic RPC system

    Project mention: Akka became the de-facto solution for Scala web development? | reddit.com/r/scala | 2021-09-18

    Finagle, possibly with Finch on top.

  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo Gatling

    Modern Load Testing as Code

    Project mention: Gatling will support Kotlin and Java | news.ycombinator.com | 2021-09-23
  • GitHub repo dotty

    The Scala 3 compiler, also known as Dotty.

    Project mention: In which version did Scala add @static to the language? | reddit.com/r/scala | 2021-08-18
  • GitHub repo Scalaz

    Principled Functional Programming in Scala

    Project mention: In Search of the Best Functional Programming Back-End: 2021 Update | dev.to | 2021-01-24

    I’ve specifically had 2 job offers internally at my company because of this language. First with Cats and Scalaz and now with ZIO, Scala has taken the best parts of Haskell, the best parts of Scala, and made it really nice to work with. You can barely see the OOP leftovers.

  • GitHub repo sbt

    sbt, the interactive build tool

    Project mention: SBT management on Apple M1 | reddit.com/r/scala | 2021-04-30

    It's an issue with the zule jvm and Java native access. A workaround is provided in this thread.

  • GitHub repo Scala.js

    Scala.js, the Scala to JavaScript compiler

    Project mention: Scala or Go: Who Wore It Better? | dev.to | 2021-09-06

    Scala became really popular with the advent of "Big Data" because functional programming lends itself so naturally to analytics, and the learning curve for modern LISPs like Haskell and Clojure is too high for too many. Apache Spark is built in Scala, and when it got big, Scala got big. Since then Scala has also become a popular language for other domains including reactive web applications and microservices with Play Framework and Akk and even the front end with Scala.js.

  • GitHub repo polynote

    A better notebook for Scala (and more)

    Project mention: Compiling/testing Jupyter notebook code with sbt? | reddit.com/r/scala | 2021-02-14

    Polynote does support .ipynb and has a feature to run notebooks from the comand line.

  • GitHub repo Scala Native

    Your favorite language gets closer to bare metal.

    Project mention: GoKart: A static analysis tool for securing Go code | news.ycombinator.com | 2021-08-18

    Scala Native probably.

    But I've never used it so not sure how mature it is.

    https://scala-native.org/

    The other Go alternative I see is D.

    Close to the metal but with high level features. Runs in a managed runtime. Creates native code.

    https://dlang.org/

  • GitHub repo BigDL

    BigDL: Distributed Deep Learning Framework for Apache Spark

    Project mention: Machine learning on JVM | reddit.com/r/scala | 2021-04-05

    Intel BigDL for Spark which again is for Spark.

  • GitHub repo Shapeless

    Generic programming for Scala

    Project mention: Delphi 11 Alexandria Has Been Released | reddit.com/r/programming | 2021-09-23

    please show me something like this: https://akka.io/ or this: https://zio.dev/ or this: https://github.com/milessabin/shapeless

  • GitHub repo ZIO

    ZIO — A type-safe, composable library for async and concurrent programming in Scala

    Project mention: Delphi 11 Alexandria Has Been Released | reddit.com/r/programming | 2021-09-23

    please show me something like this: https://akka.io/ or this: https://zio.dev/ or this: https://github.com/milessabin/shapeless

  • GitHub repo Lagom

    Reactive Microservices for the JVM

  • GitHub repo Scalatra

    Tiny Scala high-performance, async web framework, inspired by Sinatra

    Project mention: Checklist for learning Scala | dev.to | 2021-02-08

    Find the frameworks: Scalatest, Scalatra, Sangria

  • GitHub repo Twitter Util

    Wonderful reusable code from Twitter

    Project mention: Feedback on my implementation of an asynchronous semaphore | reddit.com/r/scala | 2021-02-27

    The implementation is based loosely on twitter's AsyncSemaphore

  • GitHub repo SynapseML

    Microsoft Machine Learning for Apache Spark

    Project mention: Machine learning on JVM | reddit.com/r/scala | 2021-04-05

    Microsoft ML for Spark gets you a range of powerful ML features on Spark.

  • GitHub repo Scala Exercises

    The easy way to learn Scala.

    Project mention: What should I avoid coming from Java to Scala? | reddit.com/r/scala | 2021-08-19

    Hahaha, I know when people ask for advice everyone throws in a book. But honestly, this one is the main "learn scala" book, and it's got a lot of examples so it's not as dry as the rest. Also there's a website where the examples can be done and validated in the browser: https://www.scala-exercises.org/

  • GitHub repo Scio

    A Scala API for Apache Beam and Google Cloud Dataflow.

    Project mention: ELT, Data Pipeline | dev.to | 2021-01-01

    To counter the above mentioned problem, we decided to move our data to a Pub/Sub based stream model, where we would continue to push data as it arrives. As fluentd is the primary tool being used in all our servers to gather data, rather than replacing it we leveraged its plugin architecture to use a plugin to stream data into a sink of our choosing. Initially our inclination was towards Google PubSub and Google Dataflow as our Data Scientists/Engineers use Big Query extensively and keeping the data in the same Cloud made sense. The inspiration of using these tools came from Spotify’s Event Delivery – The Road to the Cloud. We did the setup on one of our staging server with Google PubSub and Dataflow. Both didn't really work out for us as PubSub model requires a Subscriber to be available for the Topic a Publisher streams messages to, otherwise the messages are not stored. On top of it there was no way to see which messages are arriving. During this the weirdest thing that we encountered was that the Topic would be orphaned losing the subscribers when working with Dataflow. PubSub we might have managed to live with, the wall in our path was Dataflow. We started off with using SCIO from Spotify to work with Dataflow, there is a considerate lack of documentation over it and found the community to be very reserved on Github, something quite evident in the world of Scala for which they came up with a Code of Conduct for its user base to follow. Something that was required from Dataflow for us was to support batch write option to GCS, after trying our hand at Dataflow to no success to achieve that, Google's staff at StackOverflow were quite responsive and their response confirmed that it was something not available with Dataflow and streaming data to BigQuery, Datastore or Bigtable as a datastore was an option to use. The reason we didn't do that was to avoid high streaming cost to these services to store data, as majority of our jobs from the data team are based on batched hourly data. The initial proposal to the updated pipeline is shown below.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-09-23.

Index

What are some of the best open-source Scala projects in Scala? This list will help you:

Project Stars
1 Apache Spark 30,844
2 scala 13,475
3 Play 11,968
4 kafka-manager 10,344
5 Lila 9,976
6 Gitbucket 8,471
7 Finagle 8,045
8 Gatling 5,266
9 dotty 4,731
10 Scalaz 4,500
11 sbt 4,376
12 Scala.js 4,214
13 polynote 4,199
14 Scala Native 4,030
15 BigDL 3,777
16 Shapeless 3,183
17 ZIO 3,018
18 Lagom 2,574
19 Scalatra 2,525
20 Twitter Util 2,478
21 SynapseML 2,431
22 Scala Exercises 2,395
23 Scio 2,194
Find remote jobs at our new job board 99remotejobs.com. There are 25 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com