Java Apache Kafka

Open-source Java projects categorized as Apache Kafka

Top 18 Java Apache Kafka Projects

  • debezium

    Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

    Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10

    They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.

  • kafka-ui

    Open-Source Web UI for Apache Kafka Management

    Project mention: FLaNK Stack Weekly 16 October 2023 | dev.to | 2023-10-17
  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • kcctl

    A modern and intuitive command line client for Kafka Connect

  • firehose

    Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems. (by raystack)

  • kafka-ui

    Open-Source Web UI for managing Apache Kafka clusters (by kafbat)

    Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22
  • dagger

    Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data. (by raystack)

  • Kouncil

    Powerful dashboard for your Kafka. Monitor status, manage groups, topics, send messages and diagnose problems. All in one user friendly web dashboard.

    Project mention: FLaNK Stack Weekly 16 October 2023 | dev.to | 2023-10-17
  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • azure-event-hubs-for-kafka

    Azure Event Hubs for Apache Kafka Ecosystems

  • tiered-storage-for-apache-kafka

    RemoteStorageManager for Apache Kafka® Tiered Storage

    Project mention: Kafka Is Dead, Long Live Kafka | news.ycombinator.com | 2023-08-07

    I am the founder of RisingWave (http://risingwave.com/), an open-source SQL streaming database. I am happy to see the launch of Warpstream! I just reviewed the project and here's my personal opinion:

    * Apache Kafka is undoubtedly the leading product in the streaming platform space. It offers a simple yet effective API that has become the golden standard. All streaming/messaging vendors need to adhere to Kafka protocol.

    * The original Kafka only used local storage to store data, which can be extremely expensive if the data volume is large. That's why many people are advocating for the development of Kafka Tiered Storage (KIP-405: https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A...). To my best knowledge, there are at least five vendors selling Kafka or Kafka-compatible products with tiered storage support:

    -- Confluent, which builds Kora, the 10X Kafka engine: https://www.confluent.io/10x-apache-kafka/;

    -- Aiven, the open-source tiered storage Kafka (source code: https://github.com/Aiven-Open/tiered-storage-for-apache-kafk...

    -- Redpanda Data, which cuts your TCO by 6X (https://redpanda.com/platform-tco);

    -- DataStax, which commercializes Apache Pulsar (https://pulsar.apache.org/);

    -- StreamNative, which commercializes Apache Pulsar (https://pulsar.apache.org/).

    * WarpStream claims to be "built directly on top of S3," which I believe is a very aggressive approach that has the potential to drastically reduce costs, even compared to tiered storage. The potential tradeoff is system performance, especially in terms of latency. As new technology, WarpStream brings novelty, and definitely it also needs to convince users that the service is robust and reliable.

    * BYOC (Bring Your Own Cloud) is becoming the default option. Most of the vendors listed above offer BYOC, where data is stored in customers' cloud accounts, addressing concerns about data privacy and security.

    I believe WarpStream is new technology to this market, and and would encourage the team to publish some detailed numbers to confirm its performance and efficiency!

  • kryptonite-for-kafka

    Kryptonite for Kafka is a client-side 🔒 field level 🔓 cryptography library for Apache Kafka® offering a Kafka Connect SMT, ksqlDB UDFs, and a standalone HTTP API service. It's an ! UNOFFICIAL ! community project

  • opensearch-connector-for-apache-kafka

    Aiven's OpenSearch® Connector for Apache Kafka®

  • kafkaer

    Template based Kafka topic/cluster/ACL management

  • cqrs-example

    Spring Boot CQRS Example (Docker, Apache Kafka, Zookeeper, MYSQL, MongoDB)

  • MmFLaNK

    Mm FLaNK Stack (MXNet, MiNiFi, Flink, NiFi, Kafka, Kudu) for AI-IoT

    Project mention: FLaNK Stack Weekly on 26 June 2023 | dev.to | 2023-06-25

    In this interactive session, Tim will lead participants through how to best build streaming data pipelines. He will cover how to build applications from some common use cases and highlight tips, tricks, best practices and patterns. He will show how to build the easy way and then dive deep into the underlying open source technologies including Apache NiFi, Apache Flink, Apache Kafka and Apache Iceberg. If you wish to follow along, please download open source projects beforehand. You can also download this helpful streaming platform: https://docs.cloudera.com/csp-ce/latest/installation/topics/csp-ce-installing-ce.html All source code and slides will be shared for those interested in building their own FLaNK Apps. https://www.flankstack.dev/

  • camel-integration-spring-boot-kafka

    Apache Camel Integration with Spring-boot and Kafka with one application serving as a producer and the other consuming the messages

  • kafkarest-processor

    Apache NiFi 1.10.0 Processor to consume 1 Kafka message at a time, easily to tie into a REST Proxy

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2024-03-22.

Java Apache Kafka related posts

Index

What are some of the best open-source Apache Kafka projects in Java? This list will help you:

Project Stars
1 debezium 9,843
2 kafka-ui 8,416
3 kcctl 350
4 firehose 312
5 kafka-ui 266
6 dagger 254
7 Kouncil 247
8 azure-event-hubs-for-kafka 219
9 formula1-telemetry-kafka 134
10 tiered-storage-for-apache-kafka 79
11 kryptonite-for-kafka 78
12 opensearch-connector-for-apache-kafka 59
13 kafkaer 37
14 cqrs-example 23
15 MmFLaNK 6
16 camel-integration-spring-boot-kafka 4
17 cratedb-flink-jobs 2
18 kafkarest-processor 2
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com