From First Principles: Why Scala?

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • µTest

    A simple testing framework for Scala

    Let's clarify some points for folks not so familiar with Scala.

    > * Scala minor version are binary incompatible, so maintaining Scala projects is a big pain. Upgrading Spark from Scala 2.11 to Scala 2.12 was a massive undertaking for example.

    Scala just chose a strange naming scheme. Other languages would have just increased their major version instead. The scala minor version is increased every few years and not every month or so.

    > * Scala has tons of language features and lets people do crazy things in the code.

    Actually, that's not true. Or rather: compared to what language?

    Scala has surprisingly few language features, but the ones it has are very flexible and powerful. Take Kotlin for example. It has method extensions as a dedicated feature. Scala just has implicits which can be used for method extension.

    > * Scalatest is stil used by most projects and is annoying to use, as described here: https://github.com/lihaoyi/utest#why-utest. The overuse of DSLs in Scala is really annoying.

    I agree with the overuse of DSLs. Luckily that got much better, but older libraries like scalatest still suffer from that.

    > * Li's libs (os-lib, upickle, utest) have clean public interfaces, but most Scala ecosystem libs are hard to use, see the JSON alternatives for examples

    I think that just comes from using the library in a non-idiomatic way. In most applications, you will need to use the whole json anyways, and then you use (or can use) circe like that:

        {

  • Slick

    Slick (Scala Language Integrated Connection Kit) is a modern database query and access library for Scala (by slick)

    The two major SQL libraries in the Scala ecosystem these days are Doobie (https://tpolecat.github.io/doobie/) and Slick (https://scala-slick.org/).

    With Doobie you manually write your queries, and then map the results into the objects in your domain model. Nothing is generated for you. OTOH, nothing is hidden and you are free to write queries as optimized and specialized as you need. The real selling point of Doobie is a typesafe API for manipulating and combining queries, and fragments of queries, into larger wholes. This works very well when your application interfaces with a database it doesn't own.

    With Slick you get access to a DSL that lets you layout how your tables look. From there Slick offers an api that let's you treat SQL tables as-if they are basically mutable collections, with Slick handling all the SQL generation itself. You also get DDL, so that you can automate db creation and upgrades. This work very well when your application owns and controls the database it is connecting to.

    Both of these have diverged from that traditional ORM model. Slick bills itself as FRM, or Functional Relational Mapping. And Doobie is embedded queries on steroids.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • kotlin-spark-api

    This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

    Scala is nice for doing things with Spark compared to say, Java, but I really think that Kotlin can do just as well. The only advantage Scala has with Spark is the syntactic sugar. Once there is an API for Kotlin (beyond the preview version), I will be dropping Scala like a bad habit. Kotlin is also concise, has better IntelliJ support, and better Java interop.

    https://github.com/JetBrains/kotlin-spark-api

  • PyCall.jl

    Package to call Python functions from the Julia language

    For Julia there are PyCall and RCall, so the ecosystem is less of an issue than with other languages. Also, its ecosystem has been growing quickly over the last 2 years or so.

    https://github.com/JuliaPy/PyCall.jl

    https://github.com/JuliaInterop/RCall.jl

  • cf020031308

    After a number of years with Python, I started a position working in Scala about 6 months ago. I really wanted to learn something new, and become acquainted with functional programming concepts.

    I agree with most of the above. A couple of additional thoughts:

    sbt -- I still have a lot of coming up to speed to do here, but the manual is like 500 pages and it's somewhat overwhelming. There are tons of little oddities, like why can't I run `sbt --version` and instead have to do `sbt sbtVersion`?

    The functional side is fascinating -- I'm still studying the cats library. I can almost describe a Monad! It's a pretty big mountain, though, and there are times where I have doubts whether the benefits will be worth it. Would love to hear some re-assurance! ;)

    The ecosystem for microservices seems pretty closely tied to akka & lagom. These are quite complex in their own right and we've been having trouble with the latter in particular. Curious to learn about alternatives. ZIO?

    Re: DSLs. Also not a huge fan of DSLs. One refreshing thing about python is that often configuration can just be in Python itself (as in Django, for example). See also:

    [1] https://erikbern.com/2018/08/30/i-dont-want-to-learn-your-ga...

    [2] https://github.com/cf020031308/cf020031308.github.io/blob/ma...

    [3] https://twitter.com/antonycourtney/status/589238574429515777

  • Quill

    Compile-time Language Integrated Queries for Scala

    I've heard that too but I think there are at least efforts to support Scala 3.

    Having said that I think Quill looks nicer (https://getquill.io/) if you want a DSL like that.

  • ScalikeJDBC

    A tidy SQL-based DB access library for Scala developers. This library naturally wraps JDBC APIs and provides you easy-to-use APIs.

    I have had poor experiences with "fancy" SQL libraries in multiple languages when I go past basic operations. This includes Slick and Quill in Scala.

    I don't remember what the problems were with Slick that leave such a bad feeling when I hear its name -- that was 5 years ago -- but I had problems with Quill just last year. I was trying to use it to generate an efficient "in" query against a two column composite primary key, and nothing seemed to work. Since it uses macro magic, one of my attempts triggered an internal compiler error instead of normal compiler feedback.

    I ended up dropping Quill for ScalikeJDBC:

    http://scalikejdbc.org/

    It seems to be less popular/active than other libraries, but it is dead simple to use, even for developers new to Scala. I write exactly the SQL I want just like I would in psql. There is little-to-no magic [1]. I think that the only slightly magical feature I use is ensure that variable interpolation into SQL ("SQLInterpolation") prevents injection attacks.

    [1] It actually has capabilities to automatically map tables/columns into different structures and generate code for you, but my team doesn't use any of that. We just write SQL.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • treelog

    Allows logging in a tree structure so that comprehensive logging does not become incomprehensible

    The call stack is "magic" built into the language, so as soon as you're not using the "blessed" way of error handling, you lose it and need to rebuild the same functionality "by hand".

    I agree that some kind of logical call stack is a very useful thing to have, and I'd recommend implementing something along the lines of https://github.com/lancewalton/treelog that provides it.

  • dotty

    The Scala 3 compiler, also known as Dotty.

    22 is just arbitrary.

    > Is it one of those no one is going to need more than 22 arguments?

    You can work around this limit with nested tuples, or just defining Function23 yourself. Although if you're function accepts 23 arguments, you should probably refactor that anyways.

    It is also worth noting that Scala 3 drops this limitation by implementing function arguments with arrays: https://github.com/lampepfl/dotty/pull/1758

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts