librdkafka
kafka-go
Our great sponsors
librdkafka | kafka-go | |
---|---|---|
18 | 13 | |
7,292 | 7,131 | |
1.2% | 2.4% | |
8.3 | 6.7 | |
4 days ago | 4 days ago | |
C | Go | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
librdkafka
-
Do you use Rust in your professional career?
recent PR: https://github.com/confluentinc/librdkafka/pull/4275
-
JR, quality Random Data from the Command line, part I
# Kafka configuration # https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md bootstrap.servers= security.protocol=SASL_SSL sasl.mechanisms=PLAIN sasl.username= sasl.password= compression.type=gzip compression.level=9 statistics.interval.ms=1000
-
A Critical Detail about Kafka Partitioners
But what about Kafka producer clients in other languages? The excellent librdkafka project is a C/C++ implementation of Kafka clients and is widely used for non-JVM Kafka applications. Additionally, Kafka clients in other languages (Python, C#) build on top of it. The default partitioner for librdkafka uses the CRC32 hash function to get the correct partition for a key.
-
Horizontally scaling Kafka consumers with rendezvous hashing
We could have made some changes at the librdkafka level (see this), but we didn’t really want to pursue this (at least not yet).
-
Events with same key going to different partitions
You want records with the same key to always land on the same partition, so you need all the clients to use the same hashing algorithm. The easiest way to do that is to make sure the librdkafka client uses the java compatible murmur2_random hash algorithm. See “Partitioner” section here: https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md
-
Getting sum type values from a map
As my first "real world" (ish) project in Vlang, I'm trying to copy https://github.com/confluentinc/confluent-kafka-go, which is a Go wrapper for Kafka C client library, https://github.com/edenhill/librdkafka
-
Installing node-rdkafka on M1 for use with SASL
If you're using Kafka in a Node.js app, it's likely that you'll need node-rdkafka. This is a library that wraps the librdkafka library and makes it available in Node.js. According to the project's README, "All the complexity of balancing writes across partitions and managing (possibly ever-changing) brokers should be encapsulated in the library."
-
Introduction to Key Apache KafkaⓇ Concepts
# Parse the configuration. # See https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md config_parser = ConfigParser() config_parser.read_file(args.config_file) config = dict(config_parser['default']) # Create Producer instance producer = Producer(config)
-
video analytics on edge
• git clone https://github.com/edenhill/librdkafka.git
- librdkafka - the Apache Kafka C/C++ client library
kafka-go
-
book about golang and kafka
There are two main libraries that people use to write clients Confluent Kafka and segment io kafka
-
Klient - a native, statically-compiled, command line client for Kafka
Unlike the standard scripts, and many binary clients, it's a native, statically-compiled, binary. It uses segmentio/go-kafka internally, which means CGO can be disabled during compilation.
-
Is Kafka the Key? The Evolution of Highlight's Ingest
Scaling up our producers/consumers proved to be more cost-effective than adding more CPU/brokers to the Kafka cluster. To accomplish this, we used the segmentio/kafka-go client library which provides an excellent abstraction for interacting with the cluster and will handle data compression completely transparently.
-
Go EventSourcing and CQRS with PostgreSQL, Kafka, MongoDB and ElasticSearch 👋✨💫
PostgeSQL as event store database Kafka as messages broker gRPC Go implementation of gRPC Jaeger open source, end-to-end distributed tracing Prometheus monitoring and alerting Grafana for to compose observability dashboards with everything from Prometheus MongoDB MongoDB database Elasticsearch Elasticsearch client for Go. Echo web framework Kibana Kibana is data visualization dashboard software for Elasticsearch Migrate for migrations
-
Kafkagosaur - First release
I want to announce the first release of a kafkagosaur, a new Kafka client. It's built using WebAssembly and binds to the kafka-go library. The first release includes functionality to read and write Kafka messages and SASL support.
-
Switching from Celery and Python to Go
Use the Segment Kafka library, not Sarama, it’s much easier to use https://github.com/segmentio/kafka-go
-
I share my authentication server.
Kafaka - kafka-go, Debezium Outbox
-
Hunting down a C memory leak in a Go program
Segment learned quite some time ago that librdkafka-go has problems like these (and doesn’t support Contexts either), so they wrote a pure Go replacement instead. https://github.com/segmentio/kafka-go
-
Go, Kafka and gRPC clean architecture CQRS microservices with Jaeger tracing 👋🧑💻
In this article let's try to create closer to real world CQRS microservices with tracing and monitoring using: 🚀 Kafka as messages broker gRPC Go implementation of gRPC PostgreSQL as database Jaeger open source, end-to-end distributed tracing Prometheus monitoring and alerting Grafana for to compose observability dashboards with everything from Prometheus MongoDB Web and API based SMTP testing Redis Type-safe Redis client for Golang swag Swagger for Go Echo web framework
- confluent-kafka-go or Shopify/sarama
What are some alternatives?
CVE-2022-27254 - PoC for vulnerability in Honda's Remote Keyless System(CVE-2022-27254)
sarama - Sarama is a Go library for Apache Kafka. [Moved to: https://github.com/IBM/sarama]
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
Karafka - Ruby and Rails efficient multithreaded Kafka processing framework
franz-go - franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.6+. Producing, consuming, transacting, administrating, etc.
rsyslog - a Rocket-fast SYStem for LOG processing
retry-go - Simple golang library for retry mechanism
rust-kafka-101 - Getting started with Rust and Kafka
kafka-rust - Rust client for Apache Kafka
rskafka - A minimal Rust client for Apache Kafka
cobra - A Commander for modern Go CLI interactions