spark-cassandra-connector
zio-protoquill
| spark-cassandra-connector | zio-protoquill | |
|---|---|---|
| 1 | 6 | |
| 1,930 | 195 | |
| -0.1% | 1.5% | |
| 5.1 | 8.6 | |
| 7 days ago | about 19 hours ago | |
| Scala | Scala | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
spark-cassandra-connector
-
Reading from cassandra in Spark does not return all the data when using JoinWithCassandraTable
This works perfectly fine and I get all the data I'm expecting. However if I change spark.cassandra.sql.inClauseToJoinConversionThreshold(see https://github.com/datastax/spark-cassandra-connector/blob/master/doc/reference.md) to something lower like 20 which means I hit the threshold (my cross-product is 10*10=100) and JoinWithCassandraTable will be used. I suddenly do not get all the data, and on top of that I get duplicated rows for some of the data. It looks like I'm completely missing some of the partition keys, and some of the partition keys return duplicated rows (this quick-analysis might however be wrong).
zio-protoquill
-
Scala 3 migration: 7 benefits that outweigh the risks
There is: https://github.com/zio/zio-protoquill
-
Best resource to learn Scalia 3 metaprogramming
Among Scala 3-only macro codebases is proto-quill (Quill for Scala 3). Other than that most libraries will be cross-compiled Scala 2 and Scala 3 because they have to cater to both long-time Scala 2 users as well as those who already migrated.
-
Scala 3 compatible macro libraries
Proto Quill, a query DSL similar-ish to Slick, is certainly the poster child of what's possible with Scala 3 macros.
-
Fp libraries that target scala 3 exclusively?
I know that libraries like Scodec and shapeless were rewritten practically from scratch for Scala 3, taking advantage of the next syntax and internals, as well as protoquill - a Scala 3 implementation of Quill.
-
Software development topics I've changed my mind on after 6 years in the industry
I am a Scala engineer and have upstreamed a lot of open source projects to do this, (i.e. see https://github.com/getquill/protoquill/pull/17). Basically its a github action that runs concurrently to the main build, this has the following advantages
-
ZIO vs. Cats Effect for api
I've been refreshing https://github.com/getquill/protoquill nearly every day!
What are some alternatives?
deequ - Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Quill - Compile-time Language Integrated Queries for Scala
zio-http - A next-generation Scala framework for building scalable, correct, and efficient HTTP clients and servers
scodec - Scala combinator library for working with binary data
kyuubi - Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
HttpClientFactory - [Archived] Contains an opinionated factory for creating HttpClient instances. Project moved to https://github.com/aspnet/Extensions
git-issue - Git-based decentralized issue management
ducktape - Automatic and customizable compile time transformations between similar case classes and sealed traits/enums, essentially a thing that glues your code. Scala 3 only. Or is it duct 🤔
Shapeless - Generic programming for Scala
LiveTyping
spotted-leopards - Proof of concept for a cats-like library built using Dotty features
prettier - Prettier is an opinionated code formatter.