wasmCloud VS keda

There’s a lot more to go into there, but you may find https://wasmcloud.dev interesting 🙂

The wasmCloud project (https://wasmcloud.dev) has a number of areas where people can contribute. As someone else said, you can filter on "good first issue" to get an idea for what needs doing.

Webassembly is also gaining steam for server workloads due to several advantages (less overhead, better capability based security, composability, ...).

See Spin [1], Wasmcloud [2], Lunatic [3], etc.

My system is based on a distributed Webassembly runtime.

The reason for taking inspiration from Kubernetes is making deployment of distributed workloads on that runtime easy.

A nice benefit for a Kubernetes alike system is that the equivalent to controllers can be much more light-weight WASM actors that are easier to deploy and scale.

[1] https://github.com/fermyon/spin

[2] https://github.com/wasmCloud/wasmCloud

[3] https://github.com/lunatic-solutions/lunatic

It seems you want something like KEDA (https://keda.sh)

Most just utilize out of the box macro resources available in HPA.

For more advanced use cases there is keda - https://keda.sh/

Thankfully KEDA operator was already part of the cluster, and all Robin had to do was create a ScaledObject manifest targeting the Dispatch ScaleUp event, based on the rabbitmq_global_messages_received_total metric from Prometheus.

Keda

KEDA introduces event-driven scaling to Kubernetes workloads. It integrates with Kubernetes Horizontal Pod Autoscalers and can scale pods based on external metrics from services like databases and message queues (Kafka, RabbitMQ, MongoDB).

# update version 2.8.2 if required kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.8.2/keda-2.8.2.yaml

This is traditionally a AWS Lambda usecase - or an OpenFaas kind of usecase. But very recently i discovered https://keda.sh/ and it seems it is specifically meant for this in a kubernetes environment.

If you deploy the application to Amazon EKS, you can also consider using KEDA to auto-scale your consumer application based on the number of messages in the MSK topic.