Top 18 Scala Java Projects
Apache Spark - A unified analytics engine for large-scale data processingProject mention: Show HN: Box – Data Transformation Pipelines in Rust DataFusion | news.ycombinator.com | 2021-11-30
A while ago I posted a link to [Arc](https://news.ycombinator.com/item?id=26573930) a declarative method for defining repeatable data pipelines which execute against [Apache Spark](https://spark.apache.org/).
Today I would like to present a proof-of-concept implementation of the [Arc declarative ETL framework](https://arc.tripl.ai) against [Apache Datafusion](https://arrow.apache.org/datafusion/) which is an Ansi SQL (Postgres) execution engine based upon Apache Arrow and built with Rust.
The idea of providing a declarative 'configuration' language for defining data pipelines was planned from the beginning of the Arc project to allow changing execution engines without having to rewrite the base business logic (the part that is valuable to your business). Instead, by defining an abstraction layer, we can change the execution engine and run the same logic with different execution characteristics.
The benefit of the DataFusion over Apache Spark is a significant increase in speed and reduction in execution resource requirements. Even through a Docker-for-Mac inefficiency layer the same job completes in ~4 seconds with DataFusion vs ~24 seconds with Apache Spark (including JVM startup time). Without Docker-for-Mac layer end-to-end execution times of 0.5 second for the same example job (TPC-H) is possible. * the aim is not to start a benchmarking flamewar but to provide some indicative data *.
The purpose of this post is to gather feedback from the community whether you would use a tool like this, what features would be required for you to use it (MVP) or whether you would be interested in contributing to the project. I would also like to highlight the excellent work being done by the DataFusion/Arrow (and Apache) community for providing such amazing tools to us all as open source projects.
Play FrameworkProject mention: Akka became the de-facto solution for Scala web development? | reddit.com/r/scala | 2021-09-18
Play, blergh for some, not blergh for others ;)
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
A fault tolerant, protocol-agnostic RPC systemProject mention: Fun with generics: REST HTTP handler proof-of-concept | reddit.com/r/golang | 2021-11-15
This comes dangerously close to functional services as popularized by Twitter et al and encapsulated in the well known Finagle library. I don’t know but I strongly suspect /u/peterbourgon was heavily influenced by this while developing the notion of Endpoints in his reasonably well known go-kit library, although it’s significantly less general due mostly to limitations in Go’s type system.
Reactive Microservices for the JVM
Your shiny new Java/Scala build tool!Project mention: Best Scala framework / libraries out there ? | reddit.com/r/scala | 2021-10-31
Akka HTTP, Cats, Quill, ninny, Monix Observable, mill.
Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
Distributed Tracing, Metrics and Context Propagation for application running on the JVM
Run Linux Software Faster and Safer than Linux with Unikernels.
Bloop is a build server and CLI tool to compile, test and run Scala fast from any editor or build tool.Project mention: Scala 3 and Web Tech Stack | reddit.com/r/scala | 2021-07-07
A Thrift parser/generator
Web & mobile client-side akka-http sessions, with optional JWT supportProject mention: Since PlayFramework has stalled in development, any tips on migrating to akka-http? | reddit.com/r/scala | 2021-08-21
For session support with JWT, look at https://github.com/softwaremill/akka-http-session
Standalone Play WS, an async HTTP client with fluent APIProject mention: Scala or Go: Who Wore It Better? | dev.to | 2021-09-06
In this example, the code uses Play WS Standalone as a REST client to fetch JSON containing a UUID. Play WS has an asynchronous, non-blocking API based on Future, so you need to provide an ExecutionContext via Akka. That's all the boilerplate at the beginning of this example. Sometimes it will be done for you as when you use Play WS in the context of Play Framework. Nonetheless, you should be aware it has to happen somewhere.
A REPL for Apache KafkaProject mention: New version of the REPL for Apache Kafka is out! | reddit.com/r/apachekafka | 2021-03-08
Check it out! https://github.com/mmolimar/kukulcan
A scala extension for Project Reactor's Flux and Mono
ASCII-art banners in ScalaProject mention: Figlet4s v0.3.0 released | reddit.com/r/scala | 2021-04-22
I've just released version v0.3.0 of my first open source library written in Scala and FP-oriented: Figlet4s.
Bazel rules to resolve, fetch and manage 3rdparty JVM dependencies with support for multiple parallel versions of the same dependency. Powered by Coursier.Project mention: Pants vs. Bazel: Why Pants may be the right choice for your team | news.ycombinator.com | 2021-11-18
I just saw a video of twitter engineer about bazel. I know they share similar syntax and spirit, but was wondering of the details.
akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using google cloud firestore in datastore mode.
Project mention: OPAL - Open and Extensible library for analyzing and engineering Java bytecode written in Scala | reddit.com/r/programming | 2021-05-01
Scala Java related posts
Show HN: Box – Data Transformation Pipelines in Rust DataFusion
4 projects | news.ycombinator.com | 30 Nov 2021
1 project | reddit.com/r/dataengineering | 3 Nov 2021
Why should I invest in raptoreum? What makes it different
1 project | reddit.com/r/raptoreum | 25 Sep 2021
Akka became the de-facto solution for Scala web development?
6 projects | reddit.com/r/scala | 18 Sep 2021
Google Colab, Pyspark, Cassandra remote cluster combine these all together
2 projects | dev.to | 13 Sep 2021
How does Gluon's GraalVM based substrate work?
3 projects | reddit.com/r/java | 4 Sep 2021
TCP scan detected on port 9000,how dangerous is this?
1 project | reddit.com/r/HomeNetworking | 9 Aug 2021
What are some of the best open-source Java projects in Scala? This list will help you:
|17||GCP Datastore Akka Persistence Plugin||18|
Are you hiring? Post a new remote job listing for free.