Apache Kafka

Open-source projects categorized as Apache Kafka
Edit details
Language: + Java + Go + Python + Ruby + C# + C + C++ + Rust + Scala
Topics: Kafka kafka-producer kafka-connect kafka-consumer kafka-client
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Sponsored

Top 23 Apache Kafka Open-Source Projects

  • debezium

    Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

    • Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10

    They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.

  • kafka-ui

    Open-Source Web UI for Apache Kafka Management

    • Project mention: FLaNK Stack Weekly 16 October 2023 | dev.to | 2023-10-17

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • librdkafka

    The Apache Kafka C/C++ library

    • Project mention: Do you use Rust in your professional career? | /r/rust | 2023-05-09

    recent PR: https://github.com/confluentinc/librdkafka/pull/4275

  • console

    Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging. (by redpanda-data)

    • Project mention: What are your favorite tools or components in the Kafka ecosystem? | /r/apachekafka | 2023-05-31

  • Memgraph

    Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.

    • Project mention: Ask HN: Who is hiring? (March 2024) | news.ycombinator.com | 2024-03-01

    Memgraph | Staff C++ Database Engineer | REMOTE (Central/Western Europe, LatAm, or North America) https://memgraph.com/

    Memgraph is a Seed stage, open source graph database vendor. Graph DBs are a great solution for GenAI, logistics, cybersecurity and fintech so we are looking to grow aggressively this year.

    We're looking for a staff-level engineer to set technical direction, mentor junior team members, and solve some very difficult problems.

    Either DM me (the hiring manager) or apply here: https://join.com/companies/memgraph/10684850-staff-software-...

  • Karafka

    Ruby and Rails efficient multithreaded Kafka processing framework

  • rust-rdkafka

    A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka

    • Project mention: Rust Cpp Interop via Cxx, Autocxx / any best practices out there | /r/rust | 2023-06-19

    I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • cp-all-in-one

    docker-compose.yml files for cp-all-in-one , cp-all-in-one-community, cp-all-in-one-cloud, Apache Kafka Confluent Platform

    • Project mention: My local Kafka instance stuck in "auto leader balancing" | /r/apachekafka | 2023-11-22

    # https://github.com/confluentinc/cp-all-in-one/blob/7.0.1-post/cp-all-in-one/docker-compose.yml version: '3' services: zookeeper: image: confluentinc/cp-zookeeper:7.3.0 container_name: zookeeper ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-kafka:7.3.0 container_name: broker ports: - "9092:9092" depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 mongodb: container_name: mongo_c image: mongo:6.0 volumes: - ./db:/data/db ports: - "27017:27017" environment: MONGO_INITDB_ROOT_USERNAME: root MONGO_INITDB_ROOT_PASSWORD: example

  • kafkactl

    Command Line Tool for managing Apache Kafka

  • trading

    💱 Trading application written in Scala 3 that showcases an Event-Driven Architecture (EDA) and Functional Programming (FP)

    • Project mention: Scala real world projects | /r/scala | 2023-07-03

  • kafkaflow

    Apache Kafka .NET Framework to create applications simple to use and extend.

    • Project mention: FLaNK Stack 29 Jan 2024 | dev.to | 2024-01-29

  • quix-streams

    A Python library for building containerized ML and Generative AI applications with Apache Kafka.

    • Project mention: Show HN: Streaming DataFrames–a Pandas-like syntax for real-time data | news.ycombinator.com | 2024-04-23

  • KQ

    Kafka-based Job Queue for Python

  • awesome-kafka

    A list about Apache Kafka

  • transfer

    Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.

    • Project mention: Migrate mongodb Datawarehouse to snowflake | /r/snowflake | 2023-12-04

  • SlimMessageBus

    Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

  • kcctl

    A modern and intuitive command line client for Kafka Connect

  • rdkafka-ruby

    Modern and performant Kafka client library for Ruby based on librdkafka

  • firehose

    Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems. (by raystack)

  • kafka-ui

    Open-Source Web UI for managing Apache Kafka clusters (by kafbat)

    • Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22

  • dagger

    Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data. (by raystack)

  • Kouncil

    Powerful dashboard for your Kafka. Monitor status, manage groups, topics, send messages and diagnose problems. All in one user friendly web dashboard.

    • Project mention: FLaNK Stack Weekly 16 October 2023 | dev.to | 2023-10-17

  • azure-event-hubs-for-kafka

    Azure Event Hubs for Apache Kafka Ecosystems

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Apache Kafka related posts

Index

What are some of the best open-source Apache Kafka projects? This list will help you:

Project Stars
1 debezium 9,857
2 kafka-ui 8,458
3 librdkafka 7,292
4 console 3,594
5 Memgraph 2,086
6 Karafka 1,972
7 rust-rdkafka 1,477
8 cp-all-in-one 879
9 kafkactl 748
10 trading 604
11 kafkaflow 575
12 quix-streams 570
13 KQ 567
14 awesome-kafka 565
15 transfer 527
16 SlimMessageBus 436
17 kcctl 351
18 rdkafka-ruby 321
19 firehose 313
20 kafka-ui 299
21 dagger 254
22 Kouncil 247
23 azure-event-hubs-for-kafka 219

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com