Top 23 Kafka Open-Source Projects

1. pathway

   1 9 62,137 9.7 Python

   Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

   

   Project mention: GitHub's Fake Star Economy | news.ycombinator.com | 2026-04-20

2. AppSignal

   www.appsignal.com sponsored

   AppSignal knows why the f*#k it crashed. Stop vibe-debugging. Every exception, every backtrace, grouped so you see patterns, not noise.

   

3. data-engineering-zoomcamp

   2 127 46,102 9.7 Jupyter Notebook

   Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

   

   Project mention: From APIs to Warehouses: AI-Assisted Data Ingestion with dlt | dev.to | 2026-03-01

   You can sign up here: https://github.com/DataTalksClub/data-engineering-zoomcamp/

4. Apache Kafka

   3 52 33,932 9.9 Java

   Apache Kafka - A distributed event streaming platform

   

   Project mention: Going Event-Driven? Read the Bill First. | dev.to | 2026-10-09

   Event-driven architecture fixes it by deleting the waiting. Service A publishes an event (say, "Order Placed") to a broker like Kafka or Amazon EventBridge. Whoever cares about that event picks it up and reacts in their own time. Service A is already serving the next request. Nobody waits on anybody.

5. SpringBoot-Labs

   4 1 20,097 0.0 Java

   一个涵盖六个专栏：Spring Boot 2.X、Spring Cloud、Spring Cloud Alibaba、Dubbo、分布式消息队列、分布式事务的仓库。希望胖友小手一抖，右上角来个 Star，感恩 1024

   

6. Telegraf

   5 115 17,851 9.9 Go

   Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.

   

   Project mention: The Performance Battle the hardening of Vault and OWASP: What Matters | dev.to | 2026-05-07

   A common mistake we see is teams rejecting Vault hardening because they assume it will add 400ms+ p99 latency, a myth perpetuated by outdated blog posts from 2020. Our benchmarks of Vault 1.15 show that OWASP-compliant TLS 1.3 and rate limiting add only 12ms p99 latency, not 400ms. Before deciding against hardening, run the first code example (vault_owasp_benchmark.py) against your own Vault deployment to get real numbers for your workload. Use tools like https://github.com/influxdata/telegraf to collect latency metrics from production, and https://github.com/grafana/grafana to visualize percentiles over time. For read-heavy workloads (10k+ requests per second), the latency overhead is even lower (under 8ms p99) because TLS 1.3 has lower handshake overhead than TLS 1.2. We worked with a SaaS company that rejected hardening for 6 months due to latency fears, only to find after benchmarking that the overhead was 9ms p99, well within their 100ms SLA. They implemented hardening in 2 weeks, cut breach risk by 89%, and saw no customer impact. Always benchmark with your own traffic patterns, not generic numbers from the internet. Use the Go benchmark tool or Python script provided to test with your secret sizes, request patterns, and auth methods. If you see unexpected latency spikes, check for misconfigured cipher suites or rate limits that are too strict, not the hardening itself.

7. C++ Workflow

   6 9 14,426 7.2 C++

   C++ Parallel Computing and Asynchronous Networking Framework

   

8. debezium

   7 97 13,204 9.9 Java

   Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

   

   Project mention: 7 Free Tools for Data Pipeline Reconciliation and Cross-Source Validation | dev.to | 2026-05-13

   Debezium is an open-source CDC (change data capture) platform that streams database changes - inserts, updates, deletes - from supported databases (PostgreSQL, MySQL, MongoDB, SQL Server) to downstream consumers via Apache Kafka.

9. SaaSHub

   www.saashub.com sponsored

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

10. redpanda

    8 77 12,602 10.0 C++

    Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

    

    Project mention: Top Open-Source Data Engineering Tools- Unravelling the Best in 2026 | dev.to | 2025-12-10

    Redpanda

11. sarama

    9 3 12,524 9.6 Go

    Sarama is a Go library for Apache Kafka.

    

    Project mention: Kafka Consumer Health Checks: Dead or Alive | dev.to | 2025-12-13

    I’ve packaged this logic into kafka-pulse-go, a lightweight library that works with most popular Kafka clients. The core logic is decoupled from specific client implementations, with adapters provided for Sarama, segmentio/kafka-go, and Confluent’s client.

12. kafka-ui

    10 50 12,242 5.8 Java

    Open-Source Web UI for Apache Kafka Management

    

13. kubeshark

    11 19 12,098 8.5 Go

    eBPF-powered network observability for Kubernetes. Indexes L4/L7 traffic with full K8s context, decrypts TLS without keys. Queryable by AI agents via MCP and humans via dashboard.

    

    Project mention: API Traffic Analyzer for Kubernetes | news.ycombinator.com | 2026-03-09

14. kafka-manager

    12 13 11,926 0.0 Scala

    CMAK is a tool for managing Apache Kafka clusters

    

15. automq

    13 29 10,910 9.2 Java

    Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.

    

    Project mention: A 10K-star milestone for Diskless Kafka | news.ycombinator.com | 2026-06-17

16. watermill

    14 29 9,912 6.1 Go

    Building event-driven applications the easy way in Go.

    

    Project mention: How I built Upple: A modern uptime monitor with Go and React | dev.to | 2026-01-02

    I'm using Watermill for the event bus with Redis Streams as the backend. Redis Streams has this concept of consumer groups; consumers in the same group split messages between them, while different groups each receive all messages.

17. risingwave

    15 45 9,360 9.9 Rust

    Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

    

    Project mention: Building a Real-Time Crypto Arbitrage Monitoring System | dev.to | 2025-11-24

    In crypto markets, these price differences, or spreads, appear and vanish in milliseconds. If your data pipeline takes five seconds to process a batch of prices, the opportunity is already gone. This post demonstrates how to use RisingWave—an open-source real-time event streaming platform—to detect arbitrage opportunities with sub-second latency using standard SQL.

18. connect

    16 7 8,782 9.8 Go

    Fancy stream processing made operationally mundane (by redpanda-data)

    

19. kafka-go

    17 16 8,637 4.8 Go

    Kafka library in Go

    

    Project mention: Opinion: Why You Should Use NATS 2.10 Over Kafka for Edge Messaging | dev.to | 2026-04-28

20. DevOps-Bash-tools

    18 1 8,427 9.8 Shell

    1200+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..

    

21. graylog

    19 49 8,151 9.9 Java

    Free and open log management

    

    Project mention: Safety Taken to an Extreme | news.ycombinator.com | 2026-07-04

    Fable/Mythos is just garbage.

    Situation: we maintain an internal Graylog 3.1.4 branch. We like 3.1.4 because the UI is probably peak design before their UI devs made a lot of changes that made the product more difficult and less usable. It's pretty old at this point, so we've backported a ton of CVEs

    This use case is precisely what Fable would be good for. First thing we did was publicly push our fork: https://github.com/graylog2/graylog2-server . The big problem with our fork is that we haven't patched the JS side of it, we've mitigated the exploits with a proxy load balancer. For defense-in-depth, we should patch the root problems.

    Despite specifically stating to Fable what the intentions are, and making the work public, it refuses to do anything basic. It could certainly reason about the intentions here.

    There's a completely logical and simple explanation for the drama around Fable/Mythos: It can't do what Anthropic claims it can do. It's not nearly as good as the hype.

    I have a feeling in reality they discovered internally that it was cheating on exams or they have a training loop leak. All of this is garbage is an attempt to save face right before an IPO (which conveniently, they announced they might delay).

22. CAP

    20 8 7,112 6.6 C#

    Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern (by dotnetcore)

    

23. flexprice

    21 5 6,877 10.0 Go

    Usage-based pricing and billing for developers 🔓 Cloud or self-hosted ⚙️ No-code UI 💰 Realtime usage metering 🎟 Credits & top-ups 🔑 Control feature access

    

    Project mention: How to calculate customer usage for billing | dev.to | 2026-09-21

    Run it locally and push a few events through the chain before deciding whether to build it yourself: github.com/flexprice/flexprice.

24. Faust

    22 8 6,823 1.4 Python

    Python Stream Processing

    

25. flink-cdc

    23 6 6,482 9.3 Java

    Flink CDC is a streaming data integration tool

    

    Project mention: Flink CDC from the Trenches: Handling JSON in Pipelines with UDFs | dev.to | 2025-12-16

    A critically important class that Flink CDC ships to perform type conversions internally is the DataTypeConverter, which you can find the source code for right here.

26. SaaSHub

    www.saashub.com sponsored

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    

Kafka related posts

• Clean Architecture in Python with FastAPI, FastStream, and Dishka

  4 projects | dev.to | 11 Oct 2026

• kafkactl: kubectl-Style Commands for Kafka

  3 projects | dev.to | 10 Oct 2026

• Going Event-Driven? Read the Bill First.

  1 project | dev.to | 9 Oct 2026

• Show HN: OneTUI 0.2.x k9s for your databases and message queues

  1 project | news.ycombinator.com | 5 Oct 2026

• Kafka Just Learned to Be a Queue. Postgres Has Been One All Along.

  5 projects | dev.to | 5 Oct 2026

• The Complete Picture: Azure Container Apps + Dapr Reference Architecture for .NET

  5 projects | dev.to | 1 Oct 2026

• How to calculate customer usage for billing

  1 project | dev.to | 21 Sep 2026
• A note from our sponsor - SaaSHub
  www.saashub.com | 12 Oct 2026

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

AppSignal knows why the f*#k it crashed.

Stop vibe-debugging. Every exception, every backtrace, grouped so you see patterns, not noise.

www.appsignal.com