runtime VS keda

If you're not already familiar with Acorn, I recommend checking out the official docs to learn more about it! Basically, it is an app platform that makes it easy to deploy cloud applications and their dependencies by describing them in a simple Acornfile. Instead of configuring all of the required Kubernetes manifests to run our application in the cloud, we can just use an Acornfile.

Using open-source command-line tools Draft and Acorn, we'll containerize and deploy to AKS in just a few steps! Let's go 🚀

Home | Acorn Docs

What does the following refer to in their GitHub readme?

> Caution: Consuming this project can expose you to chemicals, which are known to the State of California to cause cancer and birth defects or other reproductive harm.

https://github.com/acorn-io/acorn/blob/8a474b9e593956c2ad56b...

It seems you want something like KEDA (https://keda.sh)

Most just utilize out of the box macro resources available in HPA.

For more advanced use cases there is keda - https://keda.sh/

Thankfully KEDA operator was already part of the cluster, and all Robin had to do was create a ScaledObject manifest targeting the Dispatch ScaleUp event, based on the rabbitmq_global_messages_received_total metric from Prometheus.

Keda

KEDA introduces event-driven scaling to Kubernetes workloads. It integrates with Kubernetes Horizontal Pod Autoscalers and can scale pods based on external metrics from services like databases and message queues (Kafka, RabbitMQ, MongoDB).

# update version 2.8.2 if required kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.8.2/keda-2.8.2.yaml

This is traditionally a AWS Lambda usecase - or an OpenFaas kind of usecase. But very recently i discovered https://keda.sh/ and it seems it is specifically meant for this in a kubernetes environment.

If you deploy the application to Amazon EKS, you can also consider using KEDA to auto-scale your consumer application based on the number of messages in the MSK topic.