SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 kafka-connect Open-Source Projects
-
debezium
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
demo-scene
👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
kafka-docker-playground
🐳✨ Fully automated Apache Kafka® and Confluent Docker based examples // 👷♂️ Easily build examples or reproduction models
-
-
langstream
LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka. (by LangStream)
-
-
-
kafka-connect-file-pulse
🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
-
-
ksql-udf-deep-learning-mqtt-iot
Deep Learning UDF for KSQL for Streaming Anomaly Detection of MQTT IoT Sensor Data
-
-
-
-
streaming-ops
Simulated production environment running Kubernetes targeting Apache Kafka and Confluent components on Confluent Cloud. Managed by declarative infrastructure and GitOps.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10They manage data in the application layer and your original data stays where it is. This way data consistency is no longer an issue as it was with streaming databases. You can use Change Data Capture (CDC) services like Debezium by directly connecting to your primary database, doing computational work, and saving the result back or sending real-time data to output streams.
Project mention: Knative switchboard series, part 1. Setup Knative Eventing with Kafka from scratch, scale based on events volume, and monitor | dev.to | 2024-01-04Knative dashboards together with Kafka's dashboards it sheds light on almost any aspect of what's going on in the system.
Here’s a great place to dig into things: https://developer.confluent.io
Project mention: Difficulty configuring log4j when deploying code as plugin for an app | /r/CodingHelp | 2023-10-27I am working on a custom Kafka-Mongo sink connector (specifically, a custom WriteModelStrategy to be used with the official Mongo sink connector here: https://github.com/mongodb/mongo-kafka ). My code is not a standalone, executable Java application but rather a JAR that augments the functionality of another Java application.
Project mention: Kafka Connect Filepulse 2.13.0 is now available! This version includes support for SFTP and Alibaba OSS. It also contains many bug fixes and improvements. 🚀 | /r/apachekafka | 2023-09-15
Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22
Project mention: Saving Kafka headers as mongo fields with mongo sink connector | /r/dotnet | 2023-07-15But I get the following error: "MAP is not supported type", even though I'm sending as a string. Looking at the source code (https://github.com/jcustenborder/kafka-connect-transform-common/blob/master/src/main/java/com/github/jcustenborder/kafka/connect/transform/common/BaseTransformation.java) it blows at line 120, because it thinks it's Map type. I think the issue with this is because the header it not being correctly sent, because Azure EventHubs sends with Amqp protocol that adds a few bytes in the string...
Project mention: Dashdive (YC W23) – Track your cloud costs precisely | news.ycombinator.com | 2024-01-29We actually use Kafka rather than Kinesis, although they're very similar. For writing to ClickHouse from Kafka, we use the ClickHouse Kafka sink connector: https://github.com/ClickHouse/clickhouse-kafka-connect.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
kafka-connect related posts
-
debezium VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
- How the heck do I validate records with this kind of data??
- Difficulty configuring log4j when deploying code as plugin for an app
- Kafka Connect Filepulse 2.13.0 is now available! This version includes support for SFTP and Alibaba OSS. It also contains many bug fixes and improvements. 🚀
- Managed Relational Databases with AWS RDS and Aurora
- Real-time Data Processing Pipeline With MongoDB, Kafka, Debezium And RisingWave
- Postgresql to hadoop in real time
-
A note from our sponsor - SaaSHub
www.saashub.com | 24 Apr 2024
Index
What are some of the best open-source kafka-connect projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | debezium | 9,857 |
| 2 | kafka-ui | 8,458 |
| 3 | ksql | 5,811 |
| 4 | strimzi-kafka-operator | 4,445 |
| 5 | demo-scene | 1,453 |
| 6 | DataEngineeringProject | 985 |
| 7 | kafka-connect-elasticsearch | 744 |
| 8 | kafka-docker-playground | 593 |
| 9 | kafka-connect-ui | 495 |
| 10 | langstream | 351 |
| 11 | kcctl | 350 |
| 12 | mongo-kafka | 321 |
| 13 | kafka-connect-file-pulse | 305 |
| 14 | kafka-ui | 291 |
| 15 | ksql-udf-deep-learning-mqtt-iot | 284 |
| 16 | clickhouse-sink-connector | 168 |
| 17 | kafka-connect-transform-common | 146 |
| 18 | clickhouse-kafka-connect | 130 |
| 19 | kafka-connect-twitter | 126 |
| 20 | snowflake-kafka-connector | 122 |
| 21 | kukulcan | 114 |
| 22 | terraform-provider-kafka-connect | 105 |
| 23 | streaming-ops | 84 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com