Java Kafka

Open-source Java projects categorized as Kafka Edit details

Top 23 Java Kafka Projects

  • Apache Kafka

    Mirror of Apache Kafka

    Project mention: How-to-Guide: Contributing to Open Source | reddit.com/r/dataengineering | 2022-06-11

    Apache Kafka

  • SpringBoot-Labs

    一个涵盖六个专栏:Spring Boot 2.X、Spring Cloud、Spring Cloud Alibaba、Dubbo、分布式消息队列、分布式事务的仓库。希望胖友小手一抖,右上角来个 Star,感恩 1024

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • Thingsboard

    Open-source IoT Platform - Device management, data collection, processing and visualization.

    Project mention: Zelensky: “Ukraine has a lot to offer the world, products and services created by Ukrainians. Ukrainian companies – join. World – use. Tell your foreign friends about it. Spend with Ukraine to stand for Ukraine.“ | reddit.com/r/ukraine | 2022-06-23

    The company I work for uses https://thingsboard.io and the team from Ukraine that built it are awesome and the company is still responding even though they are going through this hell of a war.

  • debezium

    Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

    Project mention: Hard things in Computer Science | dev.to | 2022-06-27

    While working on this problem, I discovered Change-Data-Capture. The idea behind CDC is to send updates to a single store and stream the diffs of the new state to the other one. To implement it by oneself is not trivial. I'd recommend using an existing product: I've used Debezium successfully in the past for my demos.

  • graylog

    Free and open source log management

    Project mention: ELK Stack or an alternative in 2022? | reddit.com/r/sysadmin | 2022-06-09

    That's not ELK, you need a logging observability tool. For example Grafana Loki or maybe Graylog, which uses elastic, but is not ELK..

  • ksql

    The database purpose-built for stream processing applications.

    Project mention: PranaDB | news.ycombinator.com | 2022-04-28
  • kafdrop

    Kafka Web UI

    Project mention: What kind of monitoring tools are people using for their Kafka Deployment? | reddit.com/r/apachekafka | 2022-01-27

    Kafdrop (https://github.com/obsidiandynamics/kafdrop) is very popular and pretty solid. I'd start there. Kafka itself also comes with some cli tools and you can also extract metrics out of it via JMX/jolokia.

  • SonarLint

    Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.

  • strimzi-kafka-operator

    Apache Kafka® running on Kubernetes

    Project mention: Kafka cluster and its maintenance, what is it like? | reddit.com/r/apachekafka | 2022-06-07
  • akhq

    Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...

    Project mention: Show HN: After AKHQ (KafkaHQ), I Made Kestra Open-Source Airflow Alternative | news.ycombinator.com | 2022-03-24

    The project start as a side project (yet another side project I do the night and weekend) but was quickly promoted and used in a French Big Retail Company.

    This one trust on the project and decide to go production with Kestra. So they decide to inject some resource in order to develop some features that need and that is missing.

    But basically, not so much people for now. We are trying to start a community around the product and started to communicate around the product since few weeks only, I hope community will follow us! And I hope to succeed like on my other open source project: https://github.com/tchiotludo/akhq

  • cruise-control

    Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.

    Project mention: I'm looking to add more instances in a kafka cluster, how does kafka know that there are more instances in a cluster? How do we tell kafka that there are more instances provisioned? | reddit.com/r/apachekafka | 2022-06-22

    I used this at my last job and it made kafka mgmt a lot easier - https://github.com/linkedin/cruise-control

  • kafka-rest

    Confluent REST Proxy for Kafka

    Project mention: Kafka Rest API vs client library | reddit.com/r/apachekafka | 2022-03-21

    When you say the Rest API, are you referring to the Confluent REST Proxy?

  • secor

    Secor is a service implementing Kafka log persistence

    Project mention: Storing kakfa messages to dynamoDb | reddit.com/r/apachekafka | 2022-02-11

    Since you are trying to archive this data, a simpler solution is to use something like secor to archive the data in kafka to S3. It’s much cheaper than dynamodb too.

  • schema-registry

    Confluent Schema Registry for Kafka

    Project mention: Testing a Kafka consumer with Avro schema messages in your Spring Boot application with Testcontainers | dev.to | 2022-02-22

    So that means we can configure the Kafka producer and consumer with an imaginary schema registry url, that only needs to start with “mock://” and you automatically get to work with the MockSchemaRegistryClient. This way you don't need to explicitly initiate the MockSchemaRegistryClient and configure everything accordingly. That also eradicates the need for the Confluent Schema Registry Container. Running the Kafka Testcontainer with the embedded Zookeeper, we no longer need an extra Zookeeper container and we are down to one Testcontainer for the messaging. This way I ended up with only two Testcontainers: Kafka and the database.

  • PretendYoureXyzzy

    A web clone of the card game Cards Against Humanity.

  • Hazelcast Jet

    Distributed Stream and Batch Processing

    Project mention: Updating data files, commits vs. pull requests | dev.to | 2021-08-15

    Hazelcast Jet

  • Nakadi

    A distributed event bus that implements a RESTful API abstraction on top of Kafka-like queues

  • kafka-connect-jdbc

    Kafka Connect connector for JDBC-compatible databases

    Project mention: How Sendoso is using Kafka for Event-Driven Architecture | dev.to | 2021-08-31

    Event sourcing is an effective architectural pattern to record changes to the application state. Event sequence is important - we need changes as they were originally applied. incoming events are first persisted into Kafka and then processed by services independently. Kafka, hence, becomes our source of truth (SOT), a data source that gives a complete picture of the data object as a whole. This, however, meant dramatic changes in our core application. Our source of truth is still the core database, but we generate events in Kafka when data gets persisted. To ensure transactional behavior, we employed Transactional Outbox pattern. Essentially, we a) created a new events table in the database, b) wrote event data in the same transaction when we update our SOT table. Kafka Connect is subsequently used to read this table and insert records in relevant Kafka topics. This ensures that we never have an inconsistent situation where data was inserted in the database but the event is not added to Kafka topic or vice versa. We evaluated a few connectors for sourcing data from Mysql (JdbcSourceConnector, and Debezium). Our scenario was supported out of the box in JdbcSourceConnector, making it possible to have one event table in Mysql where different rows could be routed to a relevant topic based on the topic field.

  • Hermes

    Fast and reliable message broker built on top of Kafka. (by allegro)

    Project mention: Got fed up with AMQP, so I wrote a HTTP message broker on top of RabbitMQ | reddit.com/r/programming | 2022-03-27

    Let's draw some parallels in the Kafka ecosystem, in which there are Confluent REST Proxy and Hermes by Allegro. You could say that since Kafka uses a binary protocol over TCP, both of these projects are "moot and useless". Despite that, Allegro aka the Poland's Amazon still maintains that project and uses it in production as a means of HTTP-based communication between their 700+ microservices arguing that it makes their development process easier, since it allows them to abstract away the intricacies of Kafka and use plain HTTP instead.

  • kafka-streams-course

    Learn Kafka Streams with several examples!

  • testcontainers-spring-boot

    Container auto-configurations for Spring Boot based integration tests

    Project mention: Building a Scalable Live Stream Chat Service with Spring WebFlux, Redis PubSub, RSocket and Auth0 | dev.to | 2021-08-20

    We will use the Playtika testcontainers-springboot library to easily spin up a Redis docker container while running the integration tests locally. This can then be used as our PubSub message broker as we test the functionality of the connection server.

  • kafka-connect-elasticsearch

    Kafka Connect Elasticsearch connector

  • zerocode

    A community-developed, free, open source, microservices API automation and load testing framework built using JUnit core runners for Http REST, SOAP, Security, Database, Kafka and much more. Zerocode Open Source enables you to create, change, orchestrate and maintain your automated test cases declaratively with absolute ease.

    Project mention: 20+ Trending and Popular Java Open Source Project | dev.to | 2022-05-10

    Zerocode

  • eventeum

    A resilient Ethereum event listener that bridges your smart contract events and backend microservices

    Project mention: Services for listening to events on smart contracts | reddit.com/r/ethereum | 2022-01-17

    I am creating a system to accept erc20 payments on my website. A few years back I came across Eventeum. I used in in a prototype (for a different purpose) and it worked great.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-06-27.

Java Kafka related posts

Index

What are some of the best open-source Kafka projects in Java? This list will help you:

Project Stars
1 Apache Kafka 22,336
2 SpringBoot-Labs 14,510
3 Thingsboard 11,930
4 debezium 7,009
5 graylog 6,162
6 ksql 5,056
7 kafdrop 3,800
8 strimzi-kafka-operator 3,272
9 akhq 2,536
10 cruise-control 2,204
11 kafka-rest 1,954
12 secor 1,769
13 schema-registry 1,765
14 PretendYoureXyzzy 1,140
15 Hazelcast Jet 998
16 Nakadi 847
17 kafka-connect-jdbc 833
18 Hermes 720
19 kafka-streams-course 696
20 testcontainers-spring-boot 670
21 kafka-connect-elasticsearch 661
22 zerocode 613
23 eventeum 403
Find remote jobs at our new job board 99remotejobs.com. There are 2 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com