SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Kafka Open-Source Projects
-
-
For Week 4 of DataTalksClub's data engineering zoomcamp, we had to install dbt and create a project. This was a formidable task. dbt is a data transformation tool that enables data analysts and engineers to transform data in a cloud analytics warehouse, BigQuery in our case. It took me a very long time to do this, and in this case I needed the homework extension.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
SpringBoot-Labs
一个涵盖六个专栏:Spring Boot 2.X、Spring Cloud、Spring Cloud Alibaba、Dubbo、分布式消息队列、分布式事务的仓库。希望胖友小手一抖,右上角来个 Star,感恩 1024
-
Thingsboard
Open-source IoT Platform - Device management, data collection, processing and visualization.
Project mention: ThingsBoard Microservices Installation Update Problem | /r/ThingsBoard | 2023-11-29Until recently I was still using TB v3.2.1, now I've set about updating the installation to the latest version. I proceeded as described at https://github.com/thingsboard/thingsboard/tree/master/docker:
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
kubeshark
The API traffic analyzer for Kubernetes providing real-time K8s protocol-level visibility, capturing and monitoring all traffic and payloads going in, out and across containers, pods, nodes and clusters. Inspired by Wireshark, purposely built for Kubernetes
Project mention: Show HN: Alaz: Open-Source, Self-Hosted, eBPF-Based K8s Monitoring | news.ycombinator.com | 2023-09-06The one similar product I had come across is Kubeshark (https://github.com/kubeshark/kubeshark). But admittedly the eBPF way seems more performant theoretically (given you can afford to have a modern-enough kernel). I'm really excited to see how this project develops out.
The eBPF-mode of innovation is pretty exciting, truly a fresh lens to building software. I'm also following Akita Software - the company building an eBPF paradigm of monitoring.
-
debezium
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.
-
openreplay
Session replay and analytics tool you can self-host. Ideal for reproducing issues, co-browsing with users and optimizing your product.
Project mention: Show HN: How Cobrowsing Works in Session Replay | news.ycombinator.com | 2024-03-14 -
redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.
-
-
-
recent PR: https://github.com/confluentinc/librdkafka/pull/4275
-
There are two main libraries that people use to write clients Confluent Kafka and segment io kafka
-
Project mention: graylog VS openobserve - a user suggested alternative | libhunt.com/r/graylog2-server | 2023-09-07
-
Project mention: Faust VS quix-streams - a user suggested alternative | libhunt.com/r/faust | 2023-12-07
-
I’ve successfully worked on projects using an asynchronous event-driven way of connecting services. I really like the decoupling of business logic and the events triggering it. I highly recommend https://github.com/ThreeDotsLabs/watermill to be more flexible when it comes to choosing the actual technology driving the async patter. It might be NATS today but requirements might change and you need to change. Watermill prepares you for this.
-
CAP
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
Project mention: Mechanism for managing faulty consumer in asynchronous event broadcast in microservices / modular monolith | /r/dotnet | 2023-05-22You might want to implement the outbox pattern. There’s a decent library that’ll help and is really easy to integrate. In particular, I’d look at using the transaction functionality
-
-
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10
To fully leverage the data is the new oil concept, companies require a special database designed to manage vast amounts of data instantly. This need has led to different database forms, including NoSQL databases, vector databases, time-series databases, graph databases, in-memory databases, and in-memory data grids. Recent years have seen the rise of cloud-based streaming databases such as RisingWave, Materialize, DeltaStream, and TimePlus. While they each have distinct commercial and technical approaches, their overarching goal remains consistent: to offer users cloud-based streaming database services.
-
Project mention: kafka-python VS quix-streams - a user suggested alternative | libhunt.com/r/kafka-python | 2023-12-07
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Kafka related posts
- Cloud Native Testing Framework
- HybridTestFramework: Cloud Native Testing Framework
- PubNubとIFTTTによるSMS通知システム
- PubNub 및 IFTTT를 사용한 SMS 알림 시스템
- Système de notification par SMS avec PubNub et IFTTT
- Wie man Ereignisse von PubNub zu RabbitMQ streamt
- Building a project in DBT
-
A note from our sponsor - SaaSHub
www.saashub.com | 28 Mar 2024
Index
What are some of the best open-source Kafka projects? This list will help you:
Project | Stars | |
---|---|---|
1 | Apache Kafka | 27,123 |
2 | data-engineering-zoomcamp | 22,027 |
3 | SpringBoot-Labs | 18,260 |
4 | Thingsboard | 15,409 |
5 | C++ Workflow | 12,255 |
6 | kafka-manager | 11,646 |
7 | sarama | 10,892 |
8 | kubeshark | 10,462 |
9 | debezium | 9,774 |
10 | openreplay | 8,713 |
11 | redpanda | 8,647 |
12 | kafka-ui | 8,266 |
13 | Benthos | 7,367 |
14 | librdkafka | 7,240 |
15 | kafka-go | 7,030 |
16 | graylog | 7,014 |
17 | Faust | 6,673 |
18 | watermill | 6,650 |
19 | CAP | 6,406 |
20 | ksql | 5,795 |
21 | materialize | 5,543 |
22 | kafka-python | 5,470 |
23 | kafdrop | 5,127 |