kafkajs VS cluster-operator

I will use the KafkaJS library to achieve it quickly. Let’s just paste the code here.

The streaming database writes back results to Kafka topics by using sink operation in real-time. Now, we need to add JavaScript code to consume and process discount messages received from Kafka and update the UI in real-time to display them. We can use the KafkaJS library, which is a popular Kafka client for Node.js to listen and consume Kafka messages.

Before we start producing and consuming events, we need to install Kafka to our API. We will use the Kafka.js client library for Node.js.

Once Kafka runs on Kube-native support that Strimzi offers, you can now start utilizing Node.js to create your streams and use libraries such as KafkaJS, a modern Apache Kafka client for Node.js.

Currently we run the SFS using podManagementPolicy: ordered, and have a list of 3 availability zones to start in, which starts pod 0 first in say us-east-1a, then pod 1 in us-east-1b, pod 2 in us-east-1c. This works fine however we've hit some scenarios where if there's multiple node restarts in multiple zones where rabbit pods are sitting the cluster can become unavailable and won't restart. As per this issue the fix is to run podManagementPolicy: parallel which will not require the lower number pods to be fully Ready before starting the higher.

RabbitMQ uses distributed deployment mechanism. You can set up instances in a highly available manner. Just like Memphis, RabbitMQ is cloud available. This allows you to run a highly available cluster on top of infrastructure such as kubernetes in the cloud.

kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml