ABRiS
Avro4s
ABRiS | Avro4s | |
---|---|---|
2 | 2 | |
222 | 716 | |
0.5% | - | |
5.5 | 5.9 | |
17 days ago | 2 days ago | |
Scala | Scala | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ABRiS
-
What tool do I use to serialize/deserialize Avro messages stored in a Kafka topic with schema registered in the schema-registry using Pyspark?
I think using Abris https://github.com/AbsaOSS/ABRiS is a good choice as it supports for the python version of spark but I cannot seem to properly understand this scala written library's integration in Pyspark(Python). It's a bit complicated as I have not used non-python written libraries in Python.
- Spark equivalent of Snowflake's variant
Avro4s
-
Scala 3 Macros: How to Read Annotations
for example https://github.com/sksamuel/avro4s - check AvroName and AvroNamespace
- Is there a JSON parsing library that supports union types?
What are some alternatives?
schema-registry - Confluent Schema Registry for Kafka
avro-codegen
dotnet-avro - An Avro implementation for .NET
ScalaPB - Protocol buffer compiler for Scala.
Schemer - Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
µPickle - uPickle: a simple, fast, dependency-free JSON & Binary (MessagePack) serialization library for Scala
registravka4s - RegistrAvKa4s - (Schema-)Registry, Avro and Kafka for Scala
scodec - Scala combinator library for working with binary data
databricks-end-to-end-streaming - End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.
msgpack - MessagePack serializer implementation for Scala / msgpack.org[Scala]
validation
ScalaBuff - the scala protocol buffers (protobuf) compiler