Top 18 Scala Java Projects
-
Project mention: Show HN: Box – Data Transformation Pipelines in Rust DataFusion | news.ycombinator.com | 2021-11-30
A while ago I posted a link to [Arc](https://news.ycombinator.com/item?id=26573930) a declarative method for defining repeatable data pipelines which execute against [Apache Spark](https://spark.apache.org/).
Today I would like to present a proof-of-concept implementation of the [Arc declarative ETL framework](https://arc.tripl.ai) against [Apache Datafusion](https://arrow.apache.org/datafusion/) which is an Ansi SQL (Postgres) execution engine based upon Apache Arrow and built with Rust.
The idea of providing a declarative 'configuration' language for defining data pipelines was planned from the beginning of the Arc project to allow changing execution engines without having to rewrite the base business logic (the part that is valuable to your business). Instead, by defining an abstraction layer, we can change the execution engine and run the same logic with different execution characteristics.
The benefit of the DataFusion over Apache Spark is a significant increase in speed and reduction in execution resource requirements. Even through a Docker-for-Mac inefficiency layer the same job completes in ~4 seconds with DataFusion vs ~24 seconds with Apache Spark (including JVM startup time). Without Docker-for-Mac layer end-to-end execution times of 0.5 second for the same example job (TPC-H) is possible. * the aim is not to start a benchmarking flamewar but to provide some indicative data *.
The purpose of this post is to gather feedback from the community whether you would use a tool like this, what features would be required for you to use it (MVP) or whether you would be interested in contributing to the project. I would also like to highlight the excellent work being done by the DataFusion/Arrow (and Apache) community for providing such amazing tools to us all as open source projects.
-
Project mention: Akka became the de-facto solution for Scala web development? | reddit.com/r/scala | 2021-09-18
Play, blergh for some, not blergh for others ;)
-
Scout APM
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
-
Project mention: Fun with generics: REST HTTP handler proof-of-concept | reddit.com/r/golang | 2021-11-15
This comes dangerously close to functional services as popularized by Twitter et al and encapsulated in the well known Finagle library. I don’t know but I strongly suspect /u/peterbourgon was heavily influenced by this while developing the notion of Endpoints in his reasonably well known go-kit library, although it’s significantly less general due mostly to limitations in Go’s type system.
-
-
Akka HTTP, Cats, Quill, ninny, Monix Observable, mill.
-
Reactive-kafka
Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
-
-
-
bloop
Bloop is a build server and CLI tool to compile, test and run Scala fast from any editor or build tool.
-
-
Project mention: Since PlayFramework has stalled in development, any tips on migrating to akka-http? | reddit.com/r/scala | 2021-08-21
For session support with JWT, look at https://github.com/softwaremill/akka-http-session
-
-
In this example, the code uses Play WS Standalone as a REST client to fetch JSON containing a UUID. Play WS has an asynchronous, non-blocking API based on Future, so you need to provide an ExecutionContext via Akka. That's all the boilerplate at the beginning of this example. Sometimes it will be done for you as when you use Play WS in the context of Play Framework. Nonetheless, you should be aware it has to happen somewhere.
-
Project mention: New version of the REPL for Apache Kafka is out! | reddit.com/r/apachekafka | 2021-03-08
Check it out! https://github.com/mmolimar/kukulcan
-
-
I've just released version v0.3.0 of my first open source library written in Scala and FP-oriented: Figlet4s.
-
bazel-multiversion
Bazel rules to resolve, fetch and manage 3rdparty JVM dependencies with support for multiple parallel versions of the same dependency. Powered by Coursier.
Project mention: Pants vs. Bazel: Why Pants may be the right choice for your team | news.ycombinator.com | 2021-11-18I just saw a video of twitter engineer about bazel. I know they share similar syntax and spirit, but was wondering of the details.
Here is the video - https://www.youtube.com/watch?v=fm6YbBLLlYo and the site https://github.com/twitter/bazel-multiversion
-
GCP Datastore Akka Persistence Plugin
akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using google cloud firestore in datastore mode.
-
Project mention: OPAL - Open and Extensible library for analyzing and engineering Java bytecode written in Scala | reddit.com/r/programming | 2021-05-01
Scala Java related posts
- Show HN: Box – Data Transformation Pipelines in Rust DataFusion
- Technology Advice
- Why should I invest in raptoreum? What makes it different
- Akka became the de-facto solution for Scala web development?
- Google Colab, Pyspark, Cassandra remote cluster combine these all together
- How does Gluon's GraalVM based substrate work?
- TCP scan detected on port 9000,how dangerous is this?
Index
What are some of the best open-source Java projects in Scala? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | Apache Spark | 31,417 |
| 2 | Play | 12,017 |
| 3 | Finagle | 8,113 |
| 4 | Lagom | 2,587 |
| 5 | Mill | 1,653 |
| 6 | Reactive-kafka | 1,353 |
| 7 | Kamon | 1,300 |
| 8 | bloop | 781 |
| 9 | Scrooge | 743 |
| 10 | akka-http-session | 432 |
| 11 | AkkaGRPC | 386 |
| 12 | Play WS | 197 |
| 13 | kukulcan | 112 |
| 14 | Reactor-Scala-Extensions | 51 |
| 15 | figlet4s | 27 |
| 16 | bazel-multiversion | 26 |
| 17 | GCP Datastore Akka Persistence Plugin | 18 |
| 18 | opal | 8 |
Are you hiring? Post a new remote job listing for free.
