kafka-go
Kafka library in Go (by segmentio)
franz-go
franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.6+. Producing, consuming, transacting, administrating, etc. (by twmb)
Our great sponsors
kafka-go | franz-go | |
---|---|---|
13 | 16 | |
7,104 | 1,469 | |
2.0% | - | |
7.3 | 9.1 | |
13 days ago | 11 days ago | |
Go | Go | |
MIT License | BSD 3-clause "New" or "Revised" License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
kafka-go
Posts with mentions or reviews of kafka-go.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-06-04.
-
book about golang and kafka
There are two main libraries that people use to write clients Confluent Kafka and segment io kafka
-
Klient - a native, statically-compiled, command line client for Kafka
Unlike the standard scripts, and many binary clients, it's a native, statically-compiled, binary. It uses segmentio/go-kafka internally, which means CGO can be disabled during compilation.
-
Is Kafka the Key? The Evolution of Highlight's Ingest
Scaling up our producers/consumers proved to be more cost-effective than adding more CPU/brokers to the Kafka cluster. To accomplish this, we used the segmentio/kafka-go client library which provides an excellent abstraction for interacting with the cluster and will handle data compression completely transparently.
-
Go EventSourcing and CQRS with PostgreSQL, Kafka, MongoDB and ElasticSearch 👋✨💫
PostgeSQL as event store database Kafka as messages broker gRPC Go implementation of gRPC Jaeger open source, end-to-end distributed tracing Prometheus monitoring and alerting Grafana for to compose observability dashboards with everything from Prometheus MongoDB MongoDB database Elasticsearch Elasticsearch client for Go. Echo web framework Kibana Kibana is data visualization dashboard software for Elasticsearch Migrate for migrations
-
Kafkagosaur - First release
I want to announce the first release of a kafkagosaur, a new Kafka client. It's built using WebAssembly and binds to the kafka-go library. The first release includes functionality to read and write Kafka messages and SASL support.
-
Switching from Celery and Python to Go
Use the Segment Kafka library, not Sarama, it’s much easier to use https://github.com/segmentio/kafka-go
-
I share my authentication server.
Kafaka - kafka-go, Debezium Outbox
-
Hunting down a C memory leak in a Go program
Segment learned quite some time ago that librdkafka-go has problems like these (and doesn’t support Contexts either), so they wrote a pure Go replacement instead. https://github.com/segmentio/kafka-go
-
Go, Kafka and gRPC clean architecture CQRS microservices with Jaeger tracing 👋🧑💻
In this article let's try to create closer to real world CQRS microservices with tracing and monitoring using: 🚀 Kafka as messages broker gRPC Go implementation of gRPC PostgreSQL as database Jaeger open source, end-to-end distributed tracing Prometheus monitoring and alerting Grafana for to compose observability dashboards with everything from Prometheus MongoDB Web and API based SMTP testing Redis Type-safe Redis client for Golang swag Swagger for Go Echo web framework
- confluent-kafka-go or Shopify/sarama
franz-go
Posts with mentions or reviews of franz-go.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-07-13.
-
Ingesting Data into OpenSearch using Apache Kafka and Go
Thankfully, Kafka and OpenSearch provide client libraries for a variety of programming languages which make it possible to write your own integration layer. This is exactly what's covered in this blog! We will make use of a custom Go application to ingest data using Go clients for Kafka and OpenSearch.
-
book about golang and kafka
You might want to gradually replace that one with https://github.com/twmb/franz-go because Shopify is looking to find a new owner for Sarama and, until or if they do, it seems to be falling behind with maintenance: https://github.com/Shopify/sarama/issues/2461 For example, they still haven’t addressed this breaking change https://github.com/Shopify/sarama/issues/2358. franz-go has worked well so far in Benthos https://github.com/benthosdev/benthos/tree/main/internal/impl/kafka and it will likely end up as the only implementation once the Sarama-based one will be deprecated
-
New Winner of Kafka Consumers: Scala to Go Journey
Can you post the code for that? Franz-go itself publishes benchmark samples for all clients here, so easy to run on your own machine and also review/adapt settings as needed: https://github.com/twmb/franz-go/tree/master/examples/bench
- Which Kafka client library should I use?
-
Use CDK to deploy a complete solution with Kafka, App Runner, EKS and DynamoDB
The MSK producer and consumer app use the franz-go library (it also supports MSK IAM authentication).
-
Getting started with MSK Serverless and AWS Lambda using Go
How to use the franz-go Go Kafka client to connect to MSK Serverless using IAM authentication.
-
Show HN: A native macOS client for Apache Kafka
Franz is a fairly obvious name for a Kafka related tool/library so hopefully it won't get confused with https://github.com/twmb/franz-go
-
Oracle DB support in Benthos
github.com/twmb/franz-go -> also early days but this is looking like a fantastic option for a kafka client library if you fancy being an early adopter. I've done the rounds on many kafka client libraries and they always seem to be a harsh compromise in some form or another, but I feel good about this one
-
Benthos - Fancy stream processing made operationally mundane
If you find the kafka input slow, try kafka_franz. It might be a bit faster, since it’s based on https://github.com/twmb/franz-go. The kafka one is based on https://github.com/Shopify/sarama. You can also write a custom input based on https://github.com/confluentinc/confluent-kafka-go, but this library relies on CGo, which can be annoying.
-
Which library/project do you wish was ported to golang?
Redpanda isn’t implemented in Go, but I second its use over the Apache server. Plenty of clients out there in pure go. https://github.com/twmb/franz-go is a personal favorite
What are some alternatives?
When comparing kafka-go and franz-go you can also consider the following projects:
sarama - Sarama is a Go library for Apache Kafka. [Moved to: https://github.com/IBM/sarama]
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
rust-rdkafka - A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka
retry-go - Simple golang library for retry mechanism
sarama - Sarama is a Go library for Apache Kafka.
kafka-rust - Rust client for Apache Kafka
go-ora - Pure go oracle client
cobra - A Commander for modern Go CLI interactions
Go-CQRS-Kafka-gRPC-Microservices - Go gRPC Kafka CQRS microservices with tracing
todo-api-microservice-example - Go microservice tutorial project using Domain Driven Design and Onion Architecture!
kafka-go vs sarama
franz-go vs sarama
kafka-go vs Confluent Kafka Golang Client
franz-go vs rust-rdkafka
kafka-go vs retry-go
franz-go vs sarama
kafka-go vs kafka-rust
franz-go vs go-ora
kafka-go vs cobra
franz-go vs Confluent Kafka Golang Client
kafka-go vs Go-CQRS-Kafka-gRPC-Microservices
franz-go vs todo-api-microservice-example