APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product. Learn more →
Top 23 Scala Big Data Projects
-
Feature transformations should be deterministic: The same input should produce the same output when the same feature definition and configuration are applied. This is what allows training, backtesting, and live inference to remain aligned. Tools such as Pandas, Spark, or feature platforms such as Feast can be used to implement that logic.
-
AppSignal
Monitoring that respects your time & budget. APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.
-
-
delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs (by delta-io)
Delta Transaction Log Protocol
-
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
Reactive-kafka
Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
-
adam
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
-
-
-
-
-
-
nussknacker
Low-code tool for automating actions on real time data | Stream processing for the users.
-
-
-
qbeast-spark
Qbeast-spark: DataSource enabling multi-dimensional indexing and efficient data sampling. Big Data, free from the unnecessary!
-
Clustering4Ever
C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.
-
Schemer
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
-
-
Spark Utils
Basic framework utilities to quickly start writing production ready Apache Spark applications
-
GridScale
Scala library for accessing various file, batch systems, job schedulers and grid middlewares.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Scala Big Data discussion
Scala Big Data related posts
-
The query succeeded. Which table state did it read?
-
delta VS sail - a user suggested alternative
2 projects | 18 Mar 2026 -
Apache Spark VS sail - a user suggested alternative
2 projects | 18 Mar 2026 -
Apache Spark VS sail - a user suggested alternative
2 projects | 18 Mar 2026 -
Delta Lake 4.0.0
-
Apache Iceberg V3 Spec new features for more efficient and flexible data lakes
-
Data Engineering with Scala: Mastering Real-Time Data Processing with Apache Flink and Google Pub/Sub
-
A note from our sponsor - AppSignal
www.appsignal.com | 9 Aug 2026
Index
What are some of the best open-source Big Data projects in Scala? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | Apache Spark | 43,788 |
| 2 | kafka-manager | 11,926 |
| 3 | delta | 8,927 |
| 4 | SynapseML | 5,237 |
| 5 | Scalding | 3,522 |
| 6 | Scio | 2,627 |
| 7 | Jupyter Scala | 1,624 |
| 8 | Reactive-kafka | 1,422 |
| 9 | adam | 1,056 |
| 10 | cudf-spark | 992 |
| 11 | H2O | 979 |
| 12 | delta-sharing | 955 |
| 13 | BIDMach | 919 |
| 14 | Gearpump | 756 |
| 15 | nussknacker | 740 |
| 16 | Vegas | 729 |
| 17 | Scoobi | 480 |
| 18 | qbeast-spark | 233 |
| 19 | Clustering4Ever | 132 |
| 20 | Schemer | 116 |
| 21 | spark-deployer | 75 |
| 22 | Spark Utils | 36 |
| 23 | GridScale | 30 |
Sponsored
Monitoring that respects your time & budget
APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.
www.appsignal.com