charts VS keda

I'm using this prometheus helm chart.

I did this using the incubator/raw chart ( https://github.com/helm/charts/tree/master/incubator/raw ), by creating a k8s Job.

I've installed Prometheus operator 0.34 (which works as expected) on cluster A (main prom)Now I want to use the federation option,I mean collect metrics from other Prometheus which is located on other K8S cluster B

So I am using the helm chart stable/traefik to deploy a reverse proxy to my cluster. I need to customise it beyond what is possible with the variables I can set for the template.

It may be that we just migrated to Bitnami's chart when the old "stable" chart was deprecated

Edit: I'm using Postgresql Stable Chart version 5.3.10

I'm using the official stable/prometheus-operator chart do deploy Prometheus with helm.

I am using using https://github.com/helm/charts/tree/master/stable/airflow helm chart and building v1.10.8 puckle/docker-airflow image with kubernetes installed on it and using that image in the helm chart,But I keep getting

If you’re looking to implement a similar pattern, it’s worth noting that tools like KEDA can help you scale workloads based on event sources such as queue length. In our case, we decided to write a custom operator instead — mainly to retain full control and avoid relying on KEDA availability or support in our specific cloud provider setup.

kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.4.0/keda-operator.yaml

Error from server (Request Entity Too Large): error when creating "https://github.com/kedacore/keda/releases/download/v2.12.1/keda-2.12.1.yaml": etcdserver: request is too large

KEDA (Kubernetes Event-Driven Autoscaling) extends Kubernetes’ scaling capabilities by allowing workloads to scale based on event-driven metrics such as message queue length, HTTP requests, or custom Prometheus queries. Unlike traditional Horizontal Pod Autoscalers (HPA) that rely solely on CPU or memory metrics, KEDA provides fine-grained control and adaptability to diverse application needs. For developers using SpinKube, KEDA enables efficient scaling of Spin apps based on application-specific metrics, making it easier to handle event-driven workloads in a Kubernetes environment. KEDA has a vast amount of built-in scalers to simplify integration with services running inside and outside of Kubernetes itself.

To address our goal, we had to look for a solution that could automatically scale resources based on HTTP request volumes. Fortunately, we quickly discovered KEDA, a Kubernetes-based Event-Driven Autoscaler, which proved to be an ideal fit for our needs. KEDA enables the scaling of any container in Kubernetes in response to event volumes from sources such as Kafka, RabbitMQ, or Prometheus. It was a perfect match, and we were thrilled to find a solution that met our requirements so seamlessly. With KEDA, the solution became surprisingly straightforward.

My instinct is that Keda is a relatively simple piece of software. There's basically one file for managing the scaledobject and scaledjob

Keda (Kubernetes Event-Driven Autoscaling) is an event-driven autoscale for Kubernetes workloads. Simply defined, it can scale an application based on the number of events needing to be handled.

> lambda apps

Yes, SST [1] uses lambdas heavily but makes it more seamless and less visible, just the place your code runs.

I’ve also found Azure Container Apps to hit the right balance. It’s kubernetes under the hood, which you don’t have to mess with at all, except that it can use KEDA [2] scaling rules to scale your containers to zero, then scale up with any of the supported KEDA scalers like when a message hits a queue.

[1] https://sst.dev/

[2] https://keda.sh/

KEDA is a Kubernetes-based autoscaler that dynamically adjusts the number of pods in your cluster based on the number of events needing to be processed. It is a lightweight, single-purpose component that integrates seamlessly with any Kubernetes cluster.

Look for example at the metrics exposed by kube state metrics: https://github.com/kubernetes/kube-state-metrics/tree/main/d...

With controllers metrics + kube state metrics about most Kubernetes resources, you can easily build alerts when a resource fails to reconcile.

> Basically, Horizontal Pod Autoscaler but with sensors which are not just "CPU"

Take a look at KEDA, it's exactly this: https://keda.sh/