descheduler VS scheduler-plugins

maybe descheduler can help? https://github.com/kubernetes-sigs/descheduler

Surprised this wasn't suggested yet, you can also use a software like the k8s Descheduler that executes periodically to rebalance your workloads across the existing nodes.

Here an example of coordination api in Go https://github.com/kubernetes-sigs/descheduler/commit/3cbae5e72ba53447a609e6001755ff395e6eeceb https://github.com/kubernetes-sigs/descheduler/commit/0a52af9ab82a52fd8c864a81f4033736f11aab34

This is something a (now former) colleague of mine pointed out: that the kubernetes descheduler can enforce a maximum lifetime[0] that sort of forces continual reboots. So if your system cannot tolerate running for a long time continously, this is one method to gracefully restart long running pods.

[0]: https://github.com/kubernetes-sigs/descheduler#podlifetime

no it does not... thats why tools like https://github.com/kubernetes-sigs/descheduler exist..

You might want to take a look at descheduler: https://github.com/kubernetes-sigs/descheduler

tldr; Knative Batch Job provider should support the respective coscheduling and kube-batch support. We had developed an in-house one for KubeFlow, from scratch. We had added Apache Arrow support into knative-serving with the respective CloudEvents interop layer, natively (i.e. secure shmem via IPC namespace, instead of message passing on the same host). We use it as a direct replacement for Apache Arrow Ballista, and had planned researching further DataFusion compat layer. Almost any modern ETL is pretty dubious without Apache Arrow.

I also looked into the scheduler plugin NodeResourcesAllocatable with the „Least“ option. This seems to be the solution to our problem, but I don‘t get how this can be applied. Our cluster is running on in-house nodes, however it is managed via Mirantis, so I don‘t know whether we could actually apply scheduler configurations.