SaaSHub helps you find the best software and product alternatives Learn more →
Top 18 Java Apache Kafka Projects
-
debezium
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
firehose
Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems. (by raystack)
-
-
dagger
Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data. (by raystack)
-
Kouncil
Powerful dashboard for your Kafka. Monitor status, manage groups, topics, send messages and diagnose problems. All in one user friendly web dashboard.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
-
kryptonite-for-kafka
Kryptonite for Kafka is a client-side 🔒 field level 🔓 cryptography library for Apache Kafka® offering a Kafka Connect SMT, ksqlDB UDFs, and a standalone HTTP API service. It's an ! UNOFFICIAL ! community project
-
-
camel-integration-spring-boot-kafka
Apache Camel Integration with Spring-boot and Kafka with one application serving as a producer and the other consuming the messages
-
cratedb-flink-jobs
This repository accompanies the article "Build a data ingestion pipeline using Kafka, Flink, and CrateDB" and the "CrateDB Community Day #2".
-
kafkarest-processor
Apache NiFi 1.10.0 Processor to consume 1 Kafka message at a time, easily to tie into a REST Proxy
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.
Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22
I am the founder of RisingWave (http://risingwave.com/), an open-source SQL streaming database. I am happy to see the launch of Warpstream! I just reviewed the project and here's my personal opinion:
* Apache Kafka is undoubtedly the leading product in the streaming platform space. It offers a simple yet effective API that has become the golden standard. All streaming/messaging vendors need to adhere to Kafka protocol.
* The original Kafka only used local storage to store data, which can be extremely expensive if the data volume is large. That's why many people are advocating for the development of Kafka Tiered Storage (KIP-405: https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A...). To my best knowledge, there are at least five vendors selling Kafka or Kafka-compatible products with tiered storage support:
-- Confluent, which builds Kora, the 10X Kafka engine: https://www.confluent.io/10x-apache-kafka/;
-- Aiven, the open-source tiered storage Kafka (source code: https://github.com/Aiven-Open/tiered-storage-for-apache-kafk...
-- Redpanda Data, which cuts your TCO by 6X (https://redpanda.com/platform-tco);
-- DataStax, which commercializes Apache Pulsar (https://pulsar.apache.org/);
-- StreamNative, which commercializes Apache Pulsar (https://pulsar.apache.org/).
* WarpStream claims to be "built directly on top of S3," which I believe is a very aggressive approach that has the potential to drastically reduce costs, even compared to tiered storage. The potential tradeoff is system performance, especially in terms of latency. As new technology, WarpStream brings novelty, and definitely it also needs to convince users that the service is robust and reliable.
* BYOC (Bring Your Own Cloud) is becoming the default option. Most of the vendors listed above offer BYOC, where data is stored in customers' cloud accounts, addressing concerns about data privacy and security.
I believe WarpStream is new technology to this market, and and would encourage the team to publish some detailed numbers to confirm its performance and efficiency!
In this interactive session, Tim will lead participants through how to best build streaming data pipelines. He will cover how to build applications from some common use cases and highlight tips, tricks, best practices and patterns. He will show how to build the easy way and then dive deep into the underlying open source technologies including Apache NiFi, Apache Flink, Apache Kafka and Apache Iceberg. If you wish to follow along, please download open source projects beforehand. You can also download this helpful streaming platform: https://docs.cloudera.com/csp-ce/latest/installation/topics/csp-ce-installing-ce.html All source code and slides will be shared for those interested in building their own FLaNK Apps. https://www.flankstack.dev/
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Java Apache Kafka related posts
-
Kafka-on-Pulsar Got Archived
-
How the heck do I validate records with this kind of data??
-
Managed Relational Databases with AWS RDS and Aurora
-
Kafka Is Dead, Long Live Kafka
-
Postgresql to hadoop in real time
-
ClickHouse Advanced Tutorial: Apply CDC from MySQL to ClickHouse
-
Debezium: Stream Changes from Your Database
-
A note from our sponsor - SaaSHub
www.saashub.com | 10 May 2024
Index
What are some of the best open-source Apache Kafka projects in Java? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | debezium | 9,907 |
| 2 | kafka-ui | 8,562 |
| 3 | kcctl | 353 |
| 4 | firehose | 312 |
| 5 | kafka-ui | 321 |
| 6 | dagger | 255 |
| 7 | Kouncil | 247 |
| 8 | azure-event-hubs-for-kafka | 219 |
| 9 | formula1-telemetry-kafka | 134 |
| 10 | tiered-storage-for-apache-kafka | 80 |
| 11 | kryptonite-for-kafka | 78 |
| 12 | opensearch-connector-for-apache-kafka | 60 |
| 13 | kafkaer | 37 |
| 14 | cqrs-example | 24 |
| 15 | MmFLaNK | 6 |
| 16 | camel-integration-spring-boot-kafka | 4 |
| 17 | cratedb-flink-jobs | 2 |
| 18 | kafkarest-processor | 2 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com