Top 23 Kafka Open-Source Projects

• Apache Kafka

  28 28,567 10.0 Java

  Mirror of Apache Kafka

  

  Project mention: Understanding Kafka Producer | dev.to | 2024-07-29

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• data-engineering-zoomcamp

  120 24,776 9.1 Jupyter Notebook

  Free Data Engineering course!

  

  Project mention: Top Github repositories for 10+ programming languages | dev.to | 2024-07-16

  Data Engineering SQL

• SpringBoot-Labs

  1 19,029 0.0 Java

  一个涵盖六个专栏：Spring Boot 2.X、Spring Cloud、Spring Cloud Alibaba、Dubbo、分布式消息队列、分布式事务的仓库。希望胖友小手一抖，右上角来个 Star，感恩 1024

  

• Thingsboard

  17 17,381 10.0 Java

  Open-source IoT Platform - Device management, data collection, processing and visualization.

  

  Project mention: ThingsBoard Microservices Installation Update Problem | /r/ThingsBoard | 2023-11-29

  Until recently I was still using TB v3.2.1, now I've set about updating the installation to the latest version. I proceeded as described at https://github.com/thingsboard/thingsboard/tree/master/docker:

• C++ Workflow

  9 13,033 9.1 C++

  C++ Parallel Computing and Asynchronous Networking Framework

  

• kafka-manager

  13 11,762 0.0 Scala

  CMAK is a tool for managing Apache Kafka clusters

  

  Project mention: FLaNK Stack Weekly 16 October 2023 | dev.to | 2023-10-17

• sarama

  2 11,453 8.6 Go

  Sarama is a Go library for Apache Kafka.

  

  Project mention: เขียน Go ต่อ Kafka ตอนที่ 1 | dev.to | 2024-06-11

• kubeshark

  17 10,956 9.4 Go

  The API traffic analyzer for Kubernetes providing real-time K8s protocol-level visibility, capturing and monitoring all traffic and payloads going in, out and across containers, pods, nodes and clusters. Inspired by Wireshark, purposely built for Kubernetes

  

  Project mention: 9 tools, libraries and extensions our developer can't live without (and why) | dev.to | 2024-06-17

  Debugging Kubernetes nodes is a nightmare. The amount of information is vast and the granularity isn’t great. Kubeshark is an API traffic analyzer for Kubernetes providing real-time K8s protocol-level visibility, capturing and monitoring all traffic and payloads going in, out and across containers, pods, nodes and clusters.

• debezium

  81 10,542 9.9 Java

  Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

  

  Project mention: Why soft deletes are evil and what to do instead | news.ycombinator.com | 2024-07-31

  This kinda misses the point, usually you don't want a "delete" in your app, you want an "archive". E.g. given the example it's often desirable to still know that a user used to be part of a group, or to later un-archive the group.

  I'd also recommend against doing audit logging on the application level if you want any kind of coverage, it's to easy to forget or work around. Database triggers or a CDC solution like Debezium [1] have more certainty and as a bonus don't require adding the audit functionality to each individual model.

  [1] https://debezium.io/

• kafka-ui

  49 9,621 5.8 Java

  Open-Source Web UI for Apache Kafka Management

  

  Project mention: How to Get Remote Code Execution in Kafka UI | news.ycombinator.com | 2024-07-22

• redpanda

  71 9,538 10.0 C++

  Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

  

  Project mention: AutoMQ Integration with Redpanda Console | dev.to | 2024-07-29

  References [1] Redpanda Console: https://redpanda.com/redpanda-console-kafka-ui [2] Redpanda: https://redpanda.com/ [3] Cluster Deployment of AutoMQ: https://docs.automq.com/zh/docs/automq-opensource/IyXrw3lHriVPdQkQLDvcPGQdnNh [4] Quick Start: https://github.com/redpanda-data/console?tab=readme-ov-file#quick-start [5] Release Redpanda Console: https://github.com/redpanda-data/console/releases/tag/v2.6.0 [6] Redpanda Console Configuration: https://docs.redpanda.com/current/reference/console/config/#example-redpanda-console-configuration-file [7] Kafdrop Github: https://github.com/obsidiandynamics/kafdrop

• connect

  3 8,111 9.8 Go

  Fancy stream processing made operationally mundane (by redpanda-data)

  

  Project mention: connect VS goka - a user suggested alternative | libhunt.com/r/redpanda-data/connect | 2024-07-23

• kafka-go

  14 7,539 5.2 Go

  Kafka library in Go

  

  Project mention: เขียน Go ต่อ Kafka ตอนที่ 1 | dev.to | 2024-06-11

• graylog

  47 7,357 10.0 Java

  Free and open log management

  

  Project mention: The IT Auditor’s Secret Sauce for SSPA Compliance | dev.to | 2024-07-20

  Graylog: Simplifies log management and analysis. Graylog

• watermill

  24 7,417 8.0 Go

  Building event-driven applications the easy way in Go.

  

  Project mention: Watermill -- a Go library for working efficiently with message streams | news.ycombinator.com | 2024-10-03

• risingwave

  28 6,903 10.0 Rust

  Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.

  

  Project mention: RisingWave: Process, manage, and analyze event streams with Postgres-style SQL | news.ycombinator.com | 2024-07-18

• Faust

  8 6,729 1.4 Python

  Python Stream Processing

  

  Project mention: Faust VS quix-streams - a user suggested alternative | libhunt.com/r/faust | 2023-12-07

• CAP

  7 6,627 8.5 C#

  Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern

  

• materialize

  123 5,739 10.0 Rust

  The Cloud Operational Data Store: use SQL to transform, deliver, and act on fast-changing data. (by MaterializeInc)

  

  Project mention: Rama on Clojure's terms, and the magic of continuation-passing style | news.ycombinator.com | 2024-10-14

• kafka-python

  8 5,599 4.5 Python

  Python client for Apache Kafka

  Project mention: kafka-python VS quix-streams - a user suggested alternative | libhunt.com/r/kafka-python | 2023-12-07

• kafdrop

  16 5,512 8.3 Java

  Kafka Web UI

  

  Project mention: Use Kafdrop to Manage AutoMQ | dev.to | 2024-07-29

  git clone https://github.com/obsidiandynamics/kafdrop.git

• Jocko

  3 4,925 0.0 Go

  Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

• strimzi-kafka-operator

  7 4,800 9.8 Java

  Apache Kafka® running on Kubernetes

  

  Project mention: Data on Kubernetes: Part 6- Strimzi for running Apache kafka | dev.to | 2024-09-16

  $ curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.43.0/strimzi-topic-operator-0.43.0.yaml" \ -o strimzi-cluster-operator.yaml

Kafka related posts

• Can AI Write Production-Ready Code? What Every Developer Should Know

  2 projects | dev.to | 3 Oct 2024

• Feldera Incremental Compute Engine

  6 projects | news.ycombinator.com | 29 Sep 2024

• Data on Kubernetes: Part 6- Strimzi for running Apache kafka

  2 projects | dev.to | 16 Sep 2024

• Warpstream Joined Confluent

  1 project | news.ycombinator.com | 14 Sep 2024

• Librdkafka: High-performance Apache Kafka C/C++ Client Library

  1 project | news.ycombinator.com | 5 Sep 2024

• Understanding the Importance of Kafka in High-Volume Data Environments

  1 project | dev.to | 30 Aug 2024

• AutoMQ: A Cloud-Native Replacement of Apache Kafka on S3

  1 project | news.ycombinator.com | 7 Aug 2024
• A note from our sponsor - SaaSHub
  www.saashub.com | 15 Oct 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com