The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 Apache Kafka Open-Source Projects
-
debezium
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
console
Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging. (by redpanda-data)
-
Memgraph
Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
cp-all-in-one
docker-compose.yml files for cp-all-in-one , cp-all-in-one-community, cp-all-in-one-cloud, Apache Kafka Confluent Platform
-
trading
💱 Trading application written in Scala 3 that showcases an Event-Driven Architecture (EDA) and Functional Programming (FP)
-
quix-streams
A Python library for building containerized ML and Generative AI applications with Apache Kafka.
-
transfer
Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.
-
SlimMessageBus
Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.
-
firehose
Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems. (by raystack)
-
dagger
Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data. (by raystack)
-
Kouncil
Powerful dashboard for your Kafka. Monitor status, manage groups, topics, send messages and diagnose problems. All in one user friendly web dashboard.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.
recent PR: https://github.com/confluentinc/librdkafka/pull/4275
Project mention: What are your favorite tools or components in the Kafka ecosystem? | /r/apachekafka | 2023-05-31
Memgraph | Staff C++ Database Engineer | REMOTE (Central/Western Europe, LatAm, or North America) https://memgraph.com/
Memgraph is a Seed stage, open source graph database vendor. Graph DBs are a great solution for GenAI, logistics, cybersecurity and fintech so we are looking to grow aggressively this year.
We're looking for a staff-level engineer to set technical direction, mentor junior team members, and solve some very difficult problems.
Either DM me (the hiring manager) or apply here: https://join.com/companies/memgraph/10684850-staff-software-...
Project mention: Rust Cpp Interop via Cxx, Autocxx / any best practices out there | /r/rust | 2023-06-19I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka
Project mention: My local Kafka instance stuck in "auto leader balancing" | /r/apachekafka | 2023-11-22# https://github.com/confluentinc/cp-all-in-one/blob/7.0.1-post/cp-all-in-one/docker-compose.yml version: '3' services: zookeeper: image: confluentinc/cp-zookeeper:7.3.0 container_name: zookeeper ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-kafka:7.3.0 container_name: broker ports: - "9092:9092" depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 mongodb: container_name: mongo_c image: mongo:6.0 volumes: - ./db:/data/db ports: - "27017:27017" environment: MONGO_INITDB_ROOT_USERNAME: root MONGO_INITDB_ROOT_PASSWORD: example
Project mention: Show HN: Streaming DataFrames–a Pandas-like syntax for real-time data | news.ycombinator.com | 2024-04-23
Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22
Apache Kafka related posts
- ElephantSQL Is Shutting Down
- Franz: Native Kafka Client for macOS, Windows and Linux
- Calling Haskell from Swift
- Show HN: Franz – A desktop client for Apache Kafka
- Kafka-on-Pulsar Got Archived
- Performance testing Kafka server using xk6-kafka
- 🔥 What is Apache Kafka and how to perform performance tests on it (Part 2) 👨🏻💻
-
A note from our sponsor - WorkOS
workos.com | 25 Apr 2024
Index
What are some of the best open-source Apache Kafka projects? This list will help you:
Project | Stars | |
---|---|---|
1 | debezium | 9,857 |
2 | kafka-ui | 8,458 |
3 | librdkafka | 7,292 |
4 | console | 3,594 |
5 | Memgraph | 2,086 |
6 | Karafka | 1,972 |
7 | rust-rdkafka | 1,477 |
8 | cp-all-in-one | 879 |
9 | kafkactl | 748 |
10 | trading | 603 |
11 | kafkaflow | 575 |
12 | quix-streams | 570 |
13 | KQ | 567 |
14 | awesome-kafka | 565 |
15 | transfer | 527 |
16 | SlimMessageBus | 433 |
17 | kcctl | 351 |
18 | rdkafka-ruby | 320 |
19 | firehose | 313 |
20 | kafka-ui | 291 |
21 | dagger | 254 |
22 | Kouncil | 247 |
23 | azure-event-hubs-for-kafka | 219 |
Sponsored