kafka-connect-datagen
Connector that generates data for demos (by confluentinc)
aws-msk-iam-auth
Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters. (by aws)
Our great sponsors
| kafka-connect-datagen | aws-msk-iam-auth | |
|---|---|---|
| 3 | 4 | |
| 172 | 130 | |
| 2.3% | 5.6% | |
| 7.2 | 6.5 | |
| 2 days ago | 5 days ago | |
| Java | Java | |
| Apache License 2.0 | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
kafka-connect-datagen
Posts with mentions or reviews of kafka-connect-datagen.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-06-20.
-
Build a data pipeline on AWS with Kafka, Kafka connect and DynamoDB
The first part will keep things relatively simple - it's all about get started easily. I will be using the Kafka Connect Datagen source connector to pump data some sample data into MSK topic and then use the AWS DynamoDB sink connector to persist that data in a DynamoDB table.
-
Getting started with Kafka Connector for Azure Cosmos DB using Docker
For the remaining scenarios, we will use a producer component to generate records. The Kafka Connect Datagen connector is our friend. It is meant for generating mock data for testing, so let’s put it to good use!
-
Generate mock data with headers
Neither kafka-connect-datagen nor voluble offer the ability to generate headers, unfortunately. It's a cool idea though.
aws-msk-iam-auth
Posts with mentions or reviews of aws-msk-iam-auth.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-06-20.
-
Using AWS IAM for client authentication
cd kafka_2.12-2.8.1/libs/ wget https://github.com/aws/aws-msk-iam-auth/releases/download/v1.1.1/aws-msk-iam-auth-1.1.1-all.jar cd ../bin/
- Why can I never get AWS tutorials to work? Am I just stupid? (rant)
- MSK tutorial does not seem to work. Specific question inside.
-
Build a data pipeline on AWS with Kafka, Kafka connect and DynamoDB
sudo -u ec2-user -i mkdir iam-auth && cd ./iam-auth wget https://github.com/aws/aws-msk-iam-auth/releases/download/1.1.0/aws-msk-iam-auth-1.1.0-all.jar cd ../ cat < /home/ec2-user/kafka/config/client-config.properties # Sets up TLS for encryption and SASL for authN. security.protocol = SASL_SSL # Identifies the SASL mechanism to use. sasl.mechanism = AWS_MSK_IAM # Binds SASL client implementation. sasl.jaas.config = software.amazon.msk.auth.iam.IAMLoginModule required; # Encapsulates constructing a SigV4 signature based on extracted credentials. # The SASL client bound by "sasl.jaas.config" invokes this class. sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler EOF export CLASSPATH=/home/ec2-user/iam-auth/aws-msk-iam-auth-1.1.0-all.jar echo "export CLASSPATH=${CLASSPATH}" | tee -a ~/.bash_profile
What are some alternatives?
When comparing kafka-connect-datagen and aws-msk-iam-auth you can also consider the following projects:
cosmosdb-kafka-connect-docker - Getting started with Kafka Connector for Azure Cosmos DB using Docker
Apache Avro - Apache Avro is a data serialization system.
kafka-connect-cosmosdb - Kafka Connect connectors for Azure Cosmos DB
Docker Compose - Define and run multi-container applications with Docker
Docker Swarm - Source repo for Docker's Documentation