raft
sarama
DISCONTINUED
Our great sponsors
raft | sarama | |
---|---|---|
7 | 20 | |
7,787 | 10,115 | |
1.3% | - | |
6.1 | 8.6 | |
1 day ago | 9 months ago | |
Go | Go | |
Mozilla Public License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
raft
-
Leader election library
Depending on your exact needs, you could try HashiCorp's Raft implementation: https://github.com/hashicorp/raft
-
Implementing a distributed key-value store on top of implementing Raft in Go
I have found the performance tests very tricky to get to pass without having any input from others. The assignment is really very unforgiving, I would wager the test suite is comparable to how commercial Raft implementations are tested (e.g. https://github.com/hashicorp/raft)
- Raft Is So Fetch: The Raft Consensus Algorithm Explained Through Mean Girls
-
Concurrency in Go is hard
While searching on GitHub, I found a pull request in the Raft implementation by Hashicorp (a distributed consensus algorithm), which we can use to demonstrate the following problem. Let’s start by showing the code (at api.go):
-
rqlite, the light distributed database built with Go and SQLite, v7.2 now with autoclustering via DNS and DNS SRV
Production-grade distributed consensus system.
-
Raft Consensus Protocol
In general Hashicorp's repos are high quality:
https://github.com/hashicorp/raft
Example application: https://github.com/Jille/raft-grpc-example
sarama
-
Ingesting Data into OpenSearch using Apache Kafka and Go
Note: Since there are multiple Go clients for Kafka (including Sarama), please make sure to consult their client documentation to confirm whether they support IAM authentication.
-
book about golang and kafka
You might want to gradually replace that one with https://github.com/twmb/franz-go because Shopify is looking to find a new owner for Sarama and, until or if they do, it seems to be falling behind with maintenance: https://github.com/Shopify/sarama/issues/2461 For example, they still haven’t addressed this breaking change https://github.com/Shopify/sarama/issues/2358. franz-go has worked well so far in Benthos https://github.com/benthosdev/benthos/tree/main/internal/impl/kafka and it will likely end up as the only implementation once the Sarama-based one will be deprecated
-
Klient - a native, statically-compiled, command line client for Kafka
I've used mainly sarama wrapped with a bit of bespoke helper libraries. Never really looked into others, just grabbed one that was actively maintained and went for it.
-
Golang bad design reference
Well, as someone who reviews a lot of code, I don't like seeing 100s of little files when a handful of logically grouped files would do. For example, this popular go project: https://github.com/Shopify/sarama is currently 256 small .go files, largely following a one class/file rule.
-
Concurrency in Go is hard
The first example is something we ran into while working on a project. Up until recently, the sarama library (Go library for Apache Kafka) contained the following piece of code (at sarama/version.go):
-
Benthos - Fancy stream processing made operationally mundane
If you find the kafka input slow, try kafka_franz. It might be a bit faster, since it’s based on https://github.com/twmb/franz-go. The kafka one is based on https://github.com/Shopify/sarama. You can also write a custom input based on https://github.com/confluentinc/confluent-kafka-go, but this library relies on CGo, which can be annoying.
- Understanding Kafka with Factorio
-
Is segmentio/kafka-go production ready ?
There are a few factors that are stopping me from using kafka-go over Shopify's sarama.
- Is there an alternative to Kafka that has better support in Rust?
-
Kafka manual commit problem
Here are our test results to find the root cause. https://github.com/Shopify/sarama/issues/1894#issuecomment-789427055
What are some alternatives?
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
kafka-go - Kafka library in Go
franz-go - franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.6+. Producing, consuming, transacting, administrating, etc.
serf - Service orchestration and management tool.
tendermint - ⟁ Tendermint Core (BFT Consensus) in Go
torrent - Full-featured BitTorrent client package and utilities
dragonboat - A feature complete and high performance multi-group Raft library in Go.
etcd - Distributed reliable key-value store for the most critical data of a distributed system [Moved to: https://github.com/etcd-io/etcd]
DHT - BitTorrent DHT Protocol && DHT Spider.
ringpop-go - Scalable, fault-tolerant application-layer sharding for Go applications
grpc-go - The Go language implementation of gRPC. HTTP/2 based RPC
librdkafka - The Apache Kafka C/C++ library