aws-graviton-getting-started VS keda

EMR serverless + Athena + Glue works for us. We are evaluating Graviton instance to further optimize stuff. AWS link if you are interested

Now we turn our attention to carbon emissions which are presented in Table 8. In the table, the AMD – CPU only and AMD – T408 server watts/hour are actual measurements on the test system during operation. To estimate the AWS server watts/hour, we reduced the CPU-only AMD number by 60%, which is the savings that Amazon claims that Graviton3 CPUs provide over other CPUs. In all three cases, we multiplied this by the number of servers, then hours, days, and years, to compute the three-year power consumption total.

https://aws.amazon.com/ec2/graviton/ https://cloud.google.com/compute/docs/instances/arm-on-compute

From the relevant product page [0]:

"AWS Graviton3 processors feature always-on memory encryption, dedicated caches for every vCPU, and support for pointer authentication."

Further reading on pointer authentication [1].

[0] https://aws.amazon.com/ec2/graviton/

[1] https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets...

Amazon makes their own Arm CPUs, like the Graviton3: https://aws.amazon.com/ec2/graviton/

Read More about Graviton Processors

According to BlackBerry, both QNX and IVY can run on EC2 instances powered by AWS’ Graviton2 processor. Graviton2 is an internally-developed processor that AWS debuted at re:Invent last year. It promises to provide up to 40% better price performance than comparable chips. "

It seems you want something like KEDA (https://keda.sh)

Most just utilize out of the box macro resources available in HPA.

For more advanced use cases there is keda - https://keda.sh/

Thankfully KEDA operator was already part of the cluster, and all Robin had to do was create a ScaledObject manifest targeting the Dispatch ScaleUp event, based on the rabbitmq_global_messages_received_total metric from Prometheus.

Keda

KEDA introduces event-driven scaling to Kubernetes workloads. It integrates with Kubernetes Horizontal Pod Autoscalers and can scale pods based on external metrics from services like databases and message queues (Kafka, RabbitMQ, MongoDB).

# update version 2.8.2 if required kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.8.2/keda-2.8.2.yaml

This is traditionally a AWS Lambda usecase - or an OpenFaas kind of usecase. But very recently i discovered https://keda.sh/ and it seems it is specifically meant for this in a kubernetes environment.

If you deploy the application to Amazon EKS, you can also consider using KEDA to auto-scale your consumer application based on the number of messages in the MSK topic.