parquet-go
franz-go
parquet-go | franz-go | |
---|---|---|
4 | 16 | |
333 | 1,486 | |
- | - | |
0.0 | 8.9 | |
9 months ago | 8 days ago | |
Go | Go | |
Apache License 2.0 | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
parquet-go
-
anyone have experience writing data to parquet files? Is there a better alternative for storing large amounts of financial tick data?
segmentio/parquet-go
-
Oracle DB support in Benthos
github.com/segmentio/parquet-go -> it's early days but his library is looking very promising for building applications that read or write parquet data, which was a major pain point not that long ago
-
ArcticDB: A Database for Observability
Finally seeing Apache Parquet and Apache Arrow used with Go efficiently and effectively!
Great job. Looking forward to exploring this more in the Prometheus and CNCF Ecosystem.
The underneath library used (https://github.com/segmentio/parquet-go) looks amazing too!
- High-performance Go library to manipulate parquet files
franz-go
-
Ingesting Data into OpenSearch using Apache Kafka and Go
Thankfully, Kafka and OpenSearch provide client libraries for a variety of programming languages which make it possible to write your own integration layer. This is exactly what's covered in this blog! We will make use of a custom Go application to ingest data using Go clients for Kafka and OpenSearch.
-
book about golang and kafka
You might want to gradually replace that one with https://github.com/twmb/franz-go because Shopify is looking to find a new owner for Sarama and, until or if they do, it seems to be falling behind with maintenance: https://github.com/Shopify/sarama/issues/2461 For example, they still haven’t addressed this breaking change https://github.com/Shopify/sarama/issues/2358. franz-go has worked well so far in Benthos https://github.com/benthosdev/benthos/tree/main/internal/impl/kafka and it will likely end up as the only implementation once the Sarama-based one will be deprecated
-
New Winner of Kafka Consumers: Scala to Go Journey
Can you post the code for that? Franz-go itself publishes benchmark samples for all clients here, so easy to run on your own machine and also review/adapt settings as needed: https://github.com/twmb/franz-go/tree/master/examples/bench
- Which Kafka client library should I use?
-
Use CDK to deploy a complete solution with Kafka, App Runner, EKS and DynamoDB
The MSK producer and consumer app use the franz-go library (it also supports MSK IAM authentication).
-
Getting started with MSK Serverless and AWS Lambda using Go
How to use the franz-go Go Kafka client to connect to MSK Serverless using IAM authentication.
-
Show HN: A native macOS client for Apache Kafka
Franz is a fairly obvious name for a Kafka related tool/library so hopefully it won't get confused with https://github.com/twmb/franz-go
-
Oracle DB support in Benthos
github.com/twmb/franz-go -> also early days but this is looking like a fantastic option for a kafka client library if you fancy being an early adopter. I've done the rounds on many kafka client libraries and they always seem to be a harsh compromise in some form or another, but I feel good about this one
-
Benthos - Fancy stream processing made operationally mundane
If you find the kafka input slow, try kafka_franz. It might be a bit faster, since it’s based on https://github.com/twmb/franz-go. The kafka one is based on https://github.com/Shopify/sarama. You can also write a custom input based on https://github.com/confluentinc/confluent-kafka-go, but this library relies on CGo, which can be annoying.
-
Which library/project do you wish was ported to golang?
Redpanda isn’t implemented in Go, but I second its use over the Apache server. Plenty of clients out there in pure go. https://github.com/twmb/franz-go is a personal favorite
What are some alternatives?
parquet-go - pure golang library for reading/writing parquet file
kafka-go - Kafka library in Go
ristretto - A high performance memory-bound Go cache
sarama - Sarama is a Go library for Apache Kafka. [Moved to: https://github.com/IBM/sarama]
encoding - Go package containing implementations of efficient encoding, decoding, and validation APIs.
rust-rdkafka - A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka
frostdb - ❄️ Coolest database around 🧊 Embeddable column database written in Go.
sarama - Sarama is a Go library for Apache Kafka.
go-ora - Pure go oracle client
Iris - The fastest HTTP/2 Go Web Framework. New, modern and easy to learn. Fast development with Code you control. Unbeatable cost-performance ratio :rocket:
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client