Top 23 Go Kafka Projects

• sarama

  1 11,012 9.2 Go

  Sarama is a Go library for Apache Kafka. (by IBM)

  

Project mention: Shopify/sarama is now IBM/sarama | /r/golang | 2023-07-11

• kubeshark

  16 10,576 9.4 Go

  The API traffic analyzer for Kubernetes providing real-time K8s protocol-level visibility, capturing and monitoring all traffic and payloads going in, out and across containers, pods, nodes and clusters. Inspired by Wireshark, purposely built for Kubernetes

  

Project mention: Show HN: Alaz: Open-Source, Self-Hosted, eBPF-Based K8s Monitoring | news.ycombinator.com | 2023-09-06

The one similar product I had come across is Kubeshark (https://github.com/kubeshark/kubeshark). But admittedly the eBPF way seems more performant theoretically (given you can afford to have a modern-enough kernel). I'm really excited to see how this project develops out.

The eBPF-mode of innovation is pretty exciting, truly a fresh lens to building software. I'm also following Akita Software - the company building an eBPF paradigm of monitoring.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Benthos

  76 7,617 9.7 Go

  Fancy stream processing made operationally mundane

  

Project mention: Ask HN: Who is hiring? (December 2023) | news.ycombinator.com | 2023-12-01

• kafka-go

  13 7,144 6.7 Go

  Kafka library in Go

  

Project mention: book about golang and kafka | /r/golang | 2023-06-04

There are two main libraries that people use to write clients Confluent Kafka and segment io kafka

• watermill

  23 6,777 7.1 Go

  Building event-driven applications the easy way in Go.

  

Project mention: Microservices communication | /r/golang | 2023-12-09

I’ve successfully worked on projects using an asynchronous event-driven way of connecting services. I really like the decoupling of business logic and the events triggering it. I highly recommend https://github.com/ThreeDotsLabs/watermill to be more flexible when it comes to choosing the actual technology driving the async patter. It might be NATS today but requirements might change and you need to change. Watermill prepares you for this.

• Jocko

  3 4,884 0.0 Go

  Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

• go-zero-looklook

  3 4,064 4.6 Go

  🔥基于go-zero(go zero) 微服务全技术栈开发最佳实践项目。Develop best practice projects based on the full technology stack of go zero (go zero) microservices.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• Gizmo

  2 3,754 0.0 Go

  A Microservice Toolkit from The New York Times

  

Project mention: GoLang — Simplifying Complexity “The Beginning” | dev.to | 2023-05-27

. Web backend (with various frameworks available) . Web Assembly (one of them is vugu framework) . Microservices (some frameworks: Go Micro, Go Kit, Gizmo, Kite) . Fragments services (Term mentioned by @jeffotoni in a microservices discussion group) . Lambdas (FaaS example) . Client Server . Terminal applications (using the tview lib) . IoT (some frameworks) . Bots (some here) . Client Applications using Web technology . Desktop using Qt+QML, Native Win Lib (example Qt, Qt widgets, Qml) . Network Applications . Protocol applications . REST Applications . SOAP Applications . GraphQL Applications . RPC Applications . TCP Applications . gRPC Applications . WebSocket Applications . GopherJS (compiles Go to JavaScript)

• console

  4 3,605 9.7 Go

  Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging. (by redpanda-data)

  

Project mention: What are your favorite tools or components in the Kafka ecosystem? | /r/apachekafka | 2023-05-31

• goka

  2 2,297 5.0 Go

  Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.

  

• vanus

  33 2,262 7.2 Go

  Vanus is a Serverless, event streaming system with processing capabilities. It easily connects SaaS, Cloud Services, and Databases to help users build next-gen Event-driven Applications.

  

• plumber

  19 2,046 7.7 Go

  A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

  

Project mention: plumber VS kaf - a user suggested alternative | libhunt.com/r/plumber | 2024-01-12

• kafka_exporter

  2 2,036 4.3 Go

  Kafka exporter for Prometheus

Project mention: Kafka Devs: Are there any important metrics that you are unable to monitor with current tools? | /r/apachekafka | 2023-05-10

For Prometheus there is the Kafka exporter.

• go-streams

  9 1,763 6.6 Go

  A lightweight stream processing library for Go

• peerdb

  7 1,816 9.9 Go

  Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

  

Project mention: PeerDB Streams – Simple, Native Postgres Change Data Capture | news.ycombinator.com | 2024-05-06

• eventhorizon

  5 1,546 3.8 Go

  Event Sourcing for Go!

  

• franz-go

  16 1,500 8.9 Go

  franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.6+. Producing, consuming, transacting, administrating, etc.

Project mention: Ingesting Data into OpenSearch using Apache Kafka and Go | dev.to | 2023-07-13

Thankfully, Kafka and OpenSearch provide client libraries for a variety of programming languages which make it possible to write your own integration layer. This is exactly what's covered in this blog! We will make use of a custom Go application to ingest data using Go clients for Kafka and OpenSearch.

• gnomock

  5 1,308 3.4 Go

  Test your code without writing mocks with ephemeral Docker containers 📦 Setup popular services with just a couple lines of code ⏱️ No bash, no yaml, only code 💻

• akvorado

  11 1,149 9.8 Go

  Flow collector, enricher and visualizer

  

Project mention: Free / OSS Tool for NetFlow Traffic Visualizer | /r/homelab | 2023-12-08

• vFlow

  1 1,063 1.8 Go

  Enterprise Network Flow Collector (IPFIX, sFlow, Netflow)

  

• go-stash

  5 1,035 6.3 Go

  go-stash is a high performance, free and open source server-side data processing pipeline that ingests data from Kafka, processes it, and then sends it to ElasticSearch.

  

• openmeter

  11 897 9.8 Go

  Cloud Metering for AI, Billing and FinOps. Collect and aggregate millions of usage events in real-time.

  

Project mention: OpenMeter – open-source Realtime Metering | news.ycombinator.com | 2024-02-09

• goflow

  6 826 2.5 Go

  The high-scalability sFlow/NetFlow/IPFIX collector used internally at Cloudflare.

  

Project mention: Integrating Cisco ASR with Splunk without Splunk Stream. | /r/Splunk | 2023-09-15

I would greatly appreciate insights and recommendations from those who have experience in this area. Additionally, I'm interested in hearing your opinions on the best tool for this task: nProbe or goflow? Your rationale behind your recommendation would be invaluable.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Go Kafka related posts

• How to tune Dapr bulk publish/subscribe for maximum throughput

  3 projects | dev.to | 14 Feb 2024

• OpenMeter – open-source Realtime Metering

  1 project | news.ycombinator.com | 9 Feb 2024

• plumber VS kaf - a user suggested alternative

  2 projects | 12 Jan 2024

• Performance testing Kafka server using xk6-kafka

  1 project | dev.to | 1 Jan 2024

• Eliminating additional bandwidth charges for multi-zone sites on Vercel

  1 project | dev.to | 22 Dec 2023

• 🔥 What is Apache Kafka and how to perform performance tests on it (Part 2) 👨🏻‍💻

  2 projects | dev.to | 18 Dec 2023

• Treat me like a web security idiot

  1 project | /r/Wordpress | 11 Dec 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 10 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Go projects with our weekly report!