rust-rdkafka
A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka (by fede1024)
franz-go
franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.6+. Producing, consuming, transacting, administrating, etc. (by twmb)
rust-rdkafka | franz-go | |
---|---|---|
8 | 16 | |
1,477 | 1,481 | |
- | - | |
8.4 | 8.9 | |
18 days ago | 7 days ago | |
Rust | Go | |
MIT License | BSD 3-clause "New" or "Revised" License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rust-rdkafka
Posts with mentions or reviews of rust-rdkafka.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-03-14.
-
Rust Cpp Interop via Cxx, Autocxx / any best practices out there
I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka
-
Trace Through a Kafka Cluster with Rust and OpenTelemetry
For this example, we're using rdkafka to build producers and consumers, because it allows us to specify custom headers for each record.
-
A Rust client library for interacting with Microsoft Airsim https://github.com/Sollimann/airsim-client
kafka
-
is there any other alternative for hadoop ecosystem that runs on rust?
You might find https://crates.io/crates/rdkafka helpful
-
Hey Rustaceans! Got an easy question? Ask here (46/2021)!
I am playing with tokio and rust-rdkafka library, following the examples like this one: https://github.com/fede1024/rust-rdkafka/blob/6fb2c37/examples/asynchronous_processing.rs
-
confluent Schema Registry and Rust
The source for the current version of the library can be found on Github. I had to increase the major version because I needed to break the API in order to support all formats supported by the current Schema Registry version. I also added the possibility to set an API key, so it can be used with Confluent Cloud, the cloud offering from Confluent. As part of the latest major refactoring it's also supporting async. This might improve performance of your app, and is also the default for the major Kafka client, more information about why you would want to use async can be found in the async book. The schemas retrieved from the Schema Registry are cached. This way the schema is only retrieved once for each id, and reused for other messages with the same id.
-
Is there an alternative to Kafka that has better support in Rust?
What's wrong with rust-rdkafka?
-
Getting started with Kafka and Rust: Part 2
This is a two-part series to help you get started with Rust and Kafka. We will be using the rust-rdkafka crate which itself is based on librdkafka (C library).
franz-go
Posts with mentions or reviews of franz-go.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-07-13.
-
Ingesting Data into OpenSearch using Apache Kafka and Go
Thankfully, Kafka and OpenSearch provide client libraries for a variety of programming languages which make it possible to write your own integration layer. This is exactly what's covered in this blog! We will make use of a custom Go application to ingest data using Go clients for Kafka and OpenSearch.
-
book about golang and kafka
You might want to gradually replace that one with https://github.com/twmb/franz-go because Shopify is looking to find a new owner for Sarama and, until or if they do, it seems to be falling behind with maintenance: https://github.com/Shopify/sarama/issues/2461 For example, they still haven’t addressed this breaking change https://github.com/Shopify/sarama/issues/2358. franz-go has worked well so far in Benthos https://github.com/benthosdev/benthos/tree/main/internal/impl/kafka and it will likely end up as the only implementation once the Sarama-based one will be deprecated
-
New Winner of Kafka Consumers: Scala to Go Journey
Can you post the code for that? Franz-go itself publishes benchmark samples for all clients here, so easy to run on your own machine and also review/adapt settings as needed: https://github.com/twmb/franz-go/tree/master/examples/bench
- Which Kafka client library should I use?
-
Use CDK to deploy a complete solution with Kafka, App Runner, EKS and DynamoDB
The MSK producer and consumer app use the franz-go library (it also supports MSK IAM authentication).
-
Getting started with MSK Serverless and AWS Lambda using Go
How to use the franz-go Go Kafka client to connect to MSK Serverless using IAM authentication.
-
Show HN: A native macOS client for Apache Kafka
Franz is a fairly obvious name for a Kafka related tool/library so hopefully it won't get confused with https://github.com/twmb/franz-go
-
Oracle DB support in Benthos
github.com/twmb/franz-go -> also early days but this is looking like a fantastic option for a kafka client library if you fancy being an early adopter. I've done the rounds on many kafka client libraries and they always seem to be a harsh compromise in some form or another, but I feel good about this one
-
Benthos - Fancy stream processing made operationally mundane
If you find the kafka input slow, try kafka_franz. It might be a bit faster, since it’s based on https://github.com/twmb/franz-go. The kafka one is based on https://github.com/Shopify/sarama. You can also write a custom input based on https://github.com/confluentinc/confluent-kafka-go, but this library relies on CGo, which can be annoying.
-
Which library/project do you wish was ported to golang?
Redpanda isn’t implemented in Go, but I second its use over the Apache server. Plenty of clients out there in pure go. https://github.com/twmb/franz-go is a personal favorite
What are some alternatives?
When comparing rust-rdkafka and franz-go you can also consider the following projects:
Kafka Rust Client - Rust client for Apache Kafka [Moved to: https://github.com/kafka-rust/kafka-rust]
kafka-go - Kafka library in Go
schema-registry - Confluent Schema Registry for Kafka
sarama - Sarama is a Go library for Apache Kafka. [Moved to: https://github.com/IBM/sarama]
sarama - Sarama is a Go library for Apache Kafka.
arewegameyet - The repository for https://arewegameyet.rs
go-ora - Pure go oracle client
kafka-rust - Rust client for Apache Kafka
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
ksqlDB-GraphQL-poc - A fairly simple setup to show how ksqlDB can be used with GraphQL.
todo-api-microservice-example - Go microservice tutorial project using Domain Driven Design and Onion Architecture!
rust-rdkafka vs Kafka Rust Client
franz-go vs kafka-go
rust-rdkafka vs schema-registry
franz-go vs sarama
rust-rdkafka vs kafka-go
franz-go vs sarama
rust-rdkafka vs arewegameyet
franz-go vs go-ora
rust-rdkafka vs kafka-rust
franz-go vs Confluent Kafka Golang Client
rust-rdkafka vs ksqlDB-GraphQL-poc
franz-go vs todo-api-microservice-example