cluster-operator VS spark-operator

Currently we run the SFS using podManagementPolicy: ordered, and have a list of 3 availability zones to start in, which starts pod 0 first in say us-east-1a, then pod 1 in us-east-1b, pod 2 in us-east-1c. This works fine however we've hit some scenarios where if there's multiple node restarts in multiple zones where rabbit pods are sitting the cluster can become unavailable and won't restart. As per this issue the fix is to run podManagementPolicy: parallel which will not require the lower number pods to be fully Ready before starting the higher.

RabbitMQ uses distributed deployment mechanism. You can set up instances in a highly available manner. Just like Memphis, RabbitMQ is cloud available. This allows you to run a highly available cluster on top of infrastructure such as kubernetes in the cloud.

kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml

I have spent days now trying to figure out a dependency issue I'm experiencing with (Py)Spark running on Kubernetes. I'm using the spark-on-k8s-operator and Spark's Google Cloud connector.

We're using https://github.com/bitnami/charts/tree/main/bitnami/spark, but I have heard good things about https://github.com/GoogleCloudPlatform/spark-on-k8s-operator as well. Hudi should not need any long running deployments as per the docs https://hudi.apache.org/docs/0.5.1/deployment/#deploying

The SparkOperator must be installed before we can use Spark on Kubernetes. Google created this operator, which is available on Github. In a nutshell, the operator is in charge of monitoring the cluster for specific events related to the spark job, as known as kind: SparkApplication

Spark-Operator (as I understand, Spark-Operator integrates well with K8s while vanilla Spark integration with K8s seems extremely complex to maintain)

Spark on K8s Operator is a project from Google that allows submitting spark applications on Kubernetes cluster using CustomResource Definition SparkApplication. It uses mutating admission webhook to modify the pod spec and add the features not officially supported by spark-submit.

In this section, you use Helm to deploy the Kubernetes Operator for Apache Spark from the incubator Chart repository. Helm is a package manager you can use to configure and deploy Kubernetes apps.

Additional details of how SparkApplications are run can be found in the design documentation.

GoogleCloudPlatform/spark-on-k8s-operator (Go): Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.