SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Kafka Open-Source Projects
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
SpringBoot-Labs
一个涵盖六个专栏：Spring Boot 2.X、Spring Cloud、Spring Cloud Alibaba、Dubbo、分布式消息队列、分布式事务的仓库。希望胖友小手一抖，右上角来个 Star，感恩 1024
-
Thingsboard
Open-source IoT Platform - Device management, data collection, processing and visualization.
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
kubeshark
The API traffic analyzer for Kubernetes providing real-time K8s protocol-level visibility, capturing and monitoring all traffic and payloads going in, out and across containers, pods, nodes and clusters. Inspired by Wireshark, purposely built for Kubernetes
-
debezium
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
-
openreplay
Session replay and analytics tool you can self-host. Ideal for reproducing issues, co-browsing with users and optimizing your product.
-
redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
-
-
-
-
-
-
-
-
CAP
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
-
risingwave
Cloud-native SQL stream processing, analytics, and management. KsqlDB and Apache Flink alternative. 🚀 10x more productive. 🚀 10x more cost-efficient.
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Apache Kafka — a distributed event streaming platform implementing a variant of the Raft consensus protocol (written in Java, integrated with Scala);
References: Data engineering zoomcamp week 6 course and homework notes: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/cohorts/2024/06-streaming
Project mention: ThingsBoard Microservices Installation Update Problem | /r/ThingsBoard | 2023-11-29Until recently I was still using TB v3.2.1, now I've set about updating the installation to the latest version. I proceeded as described at https://github.com/thingsboard/thingsboard/tree/master/docker:
Project mention: Show HN: Alaz: Open-Source, Self-Hosted, eBPF-Based K8s Monitoring | news.ycombinator.com | 2023-09-06The one similar product I had come across is Kubeshark (https://github.com/kubeshark/kubeshark). But admittedly the eBPF way seems more performant theoretically (given you can afford to have a modern-enough kernel). I'm really excited to see how this project develops out.
The eBPF-mode of innovation is pretty exciting, truly a fresh lens to building software. I'm also following Akita Software - the company building an eBPF paradigm of monitoring.
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.
Project mention: Show HN: How Cobrowsing Works in Session Replay | news.ycombinator.com | 2024-03-14
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.
recent PR: https://github.com/confluentinc/librdkafka/pull/4275
There are two main libraries that people use to write clients Confluent Kafka and segment io kafka
Project mention: graylog VS openobserve - a user suggested alternative | libhunt.com/r/graylog2-server | 2023-09-07
I’ve successfully worked on projects using an asynchronous event-driven way of connecting services. I really like the decoupling of business logic and the events triggering it. I highly recommend https://github.com/ThreeDotsLabs/watermill to be more flexible when it comes to choosing the actual technology driving the async patter. It might be NATS today but requirements might change and you need to change. Watermill prepares you for this.
Project mention: Faust VS quix-streams - a user suggested alternative | libhunt.com/r/faust | 2023-12-07
Project mention: Mechanism for managing faulty consumer in asynchronous event broadcast in microservices / modular monolith | /r/dotnet | 2023-05-22You might want to implement the outbox pattern. There’s a decent library that’ll help and is really easy to integrate. In particular, I’d look at using the transaction functionality
Project mention: Proton, a fast and lightweight alternative to Apache Flink | news.ycombinator.com | 2024-01-30How does this compare to RisingWave and Materialize?
https://github.com/risingwavelabs/risingwave
Project mention: Ask HN: How Can I Make My Front End React to Database Changes in Real-Time? | news.ycombinator.com | 2024-04-17[2] https://materialize.com/
Project mention: kafka-python VS quix-streams - a user suggested alternative | libhunt.com/r/kafka-python | 2023-12-07
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Kafka related posts
- Show HN: Streaming DataFrames–a Pandas-like syntax for real-time data
- Data Engineering Zoomcamp Week 6 - using redpanda 1
- Show HN: AutoMQ – A Cost-Effective Kafka distro that can autoscale in seconds
- Show HN: Kiwi – End-to-End Kafka Subscriptions with WebAssembly
- Final project part 5
- Franz: Native Kafka Client for macOS, Windows and Linux
- Calling Haskell from Swift
-
A note from our sponsor - SaaSHub
www.saashub.com | 26 Apr 2024
Index
What are some of the best open-source Kafka projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | Apache Kafka | 27,335 |
| 2 | data-engineering-zoomcamp | 22,446 |
| 3 | SpringBoot-Labs | 18,395 |
| 4 | Thingsboard | 15,639 |
| 5 | C++ Workflow | 12,412 |
| 6 | kafka-manager | 11,670 |
| 7 | sarama | 10,975 |
| 8 | kubeshark | 10,541 |
| 9 | debezium | 9,857 |
| 10 | openreplay | 8,812 |
| 11 | redpanda | 8,784 |
| 12 | kafka-ui | 8,458 |
| 13 | Benthos | 7,559 |
| 14 | librdkafka | 7,292 |
| 15 | kafka-go | 7,104 |
| 16 | graylog | 7,103 |
| 17 | watermill | 6,729 |
| 18 | Faust | 6,674 |
| 19 | CAP | 6,438 |
| 20 | risingwave | 6,283 |
| 21 | ksql | 5,817 |
| 22 | materialize | 5,567 |
| 23 | kafka-python | 5,481 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com