kafka-connect-datagen
Docker Compose
Our great sponsors
kafka-connect-datagen | Docker Compose | |
---|---|---|
3 | 383 | |
172 | 32,312 | |
2.3% | 1.2% | |
7.2 | 9.6 | |
3 days ago | 1 day ago | |
Java | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
kafka-connect-datagen
-
Build a data pipeline on AWS with Kafka, Kafka connect and DynamoDB
The first part will keep things relatively simple - it's all about get started easily. I will be using the Kafka Connect Datagen source connector to pump data some sample data into MSK topic and then use the AWS DynamoDB sink connector to persist that data in a DynamoDB table.
-
Getting started with Kafka Connector for Azure Cosmos DB using Docker
For the remaining scenarios, we will use a producer component to generate records. The Kafka Connect Datagen connector is our friend. It is meant for generating mock data for testing, so let’s put it to good use!
-
Generate mock data with headers
Neither kafka-connect-datagen nor voluble offer the ability to generate headers, unfortunately. It's a cool idea though.
Docker Compose
-
Hosting a simple docker-compose app with Nginx and generate a SSL with certbot on digitalocean droplet
curl -fsSL https://get.docker.com -o get-docker.sh sudo sh get-docker.sh # Install docker compose sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose # Apply executable permissions to the binary sudo chmod +x /usr/local/bin/docker-compose # Run Project docker-compose up -d
-
One Minute: Compose
Docker,
-
How to Set Up a Docker Container
This foundation now opens the door to even more powerful concepts. You can explore more advanced concepts such as container networking, streamlining the management of complex applications with Docker Compose, and how to make your application data persistent using volumes.
-
Use same Dockerfile for Dev & Production
In many projects that are containerized, especially in cases where development is also done locally with docker-compose, teams often have two Dockerfiles, 1 for Development, the other for Production. If you happen to have multiple environments like pre-prod, staging and so on, some teams could have different Dockerfiles for these environments.
-
How to Dockerise a NodeJS - TypeScript API || A Comprehensive Guide from Environment Setup to Deployment with a CI/CD Pipeline
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
-
Deploy a MongoDB Cluster With Docker Compose
You can also use Docker Compose to deploy the cluster, without having to write all the previous commands.
-
Build and Deploy a ReactJS App to AWS EC2 with Docker, NGINX, and Automate with GitHub Actions.
[ec2-user]$ sudo yum update -y [ec2-user]$ sudo yum install -y docker [ec2-user]$ sudo service docker start [ec2-user]$ sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose [ec2-user]$ sudo chmod +x /usr/local/bin/docker-compose [ec2-user]$ docker --version Docker version 20.10.23, build 7155243 [ec2-user]$ docker-compose --version Docker Compose version v2.18.1
-
MongoDB on Your Local Machine Using Docker: A Step-by-Step Guide
Docker Compose Documentation
-
Docker - Setup a local JS and Python Development environment (Part 2)
groupadd -g 997 docker gpasswd -a jenkins docker curl -L https://github.com/docker/compose/releases/tag/1.29.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose #confirm the docker-compose is installed docker-compose version
-
Docker for the Absolute Beginner
Docker Compose overview Docker compose Doc
What are some alternatives?
cosmosdb-kafka-connect-docker - Getting started with Kafka Connector for Azure Cosmos DB using Docker
supervisor - Supervisor process control system for Unix (supervisord)
Apache Avro - Apache Avro is a data serialization system.
LibreNMS-docker - LibreNMS Docker image
kafka-connect-cosmosdb - Kafka Connect connectors for Azure Cosmos DB
terraform - Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
aws-msk-iam-auth - Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Portainer - Making Docker and Kubernetes management easy.
Docker Swarm - Source repo for Docker's Documentation
Cloud-Init - unofficial mirror of Ubuntu's cloud-init
k3s - Lightweight Kubernetes
docker-cloudflared - Cloudflared proxy-dns Docker image