akri VS kubevirt

akri: AdmissionErrors and if it actually mounts something the process cant access the device for some reason.

If you're talking about hardware attached to nodes, Akri might be of interest: https://github.com/project-akri/akri

Fun hands on use-case/story. Trying to connect sensors & actuators sounds fun! It's clear that they need some kind of infra/platform that they can run on-prem, & this feels like a lock.

There's a new Dynamic Resource api in k8s 1.26 that feels potentially well suited to managing their various kinds of node-attached resources in a pretty broad/generic/cloudy way. Taking the cloud from anonymous compute/storage nodes to an all encompassing management system for whatever we have is an interesting next step. I rather doubt we'll see folks like this spend the 3x effort to pioneer use of these abstractions but over time it should start to emerge & become more regular. https://kubernetes.io/docs/concepts/scheduling-eviction/dyna...

There's also a really neat really nice CNCF Sandbox project Akri which performs semi-similar-ish functions as Dynamic Resources. One of the examples they use as a goto is managing a fleet of OVNIF cameras, like, security cameras. https://github.com/project-akri/akri

On another topic, appreciate this article talking about the mixed human/robot workflows. Good to consider! I worked at a great & interesting data-warehouse company a long time ago that had some very complex long running ETL pipelines & limited compute resources (good number of boxes... pre-cloud era!), and a lot of time-sensitivity. The company used Jira to manage the ETL end to end, with custom workflows where work would gate, process, get checked through various stages, with processors updating the ticket as work advanced or to report % complete. Intensely sharp choice not to build it ourselves, to use a tool for tracking work, even though the code (robots) we built were doing 9/10ths the work. Fun to see a similar topic discussed here.

Good problem setup, interesting view here. Would appreciate a little more technical specifics, but still fun to see.

You need something to act as the runner queue to schedule the nodes, so perhaps you could pull them into a k8s cluster and auto-scale oh the load queue. Using something like this to tag the nodes what special devices might be attached on certain nodes https://github.com/deislabs/akri

Some notes on Plex/Emby/Kodi and transcoding. If you want true transcoding with GPU acceleration, you have to have Nvidia GPU or be a k8s device plugin genius. The whole idea of mounting elastic devices in k8s is fairly new and rather complex. In the mean time transcoding is best done on a beefy device with a proper CPU (eg i7) or specifically Nvidia GPU because there are numerous pre-made plugins. I just run Plex and Emby on an old ATX gaming machine without GPU acceleration and it works totally fine. They were barely usable for just me when running on the RPis, wouldn't recommend it unless you can figure out how to mount the correct devices in the pod using a custom raspberry pi device plugin . . . lol good luck! - Arm labs device manager: https://community.arm.com/developer/research/b/articles/posts/a-smarter-device-manager-for-kubernetes-on-the-edge - Deis labs Akri device manager: https://github.com/deislabs/akri - Nvidia GPU plugin: https://github.com/NVIDIA/k8s-device-plugin

First, download virtctl for ARM: https://github.com/kubevirt/kubevirt/releases/tag/v1.1.0-alpha.0

The full list of changes can be found in the Release notes. There are performance and scalability benchmarks published for the v1.0 release.

If you're happy with k8s and longhorn, why add Proxmox as another layer underneath? Consider kubevirt ?

Have you looked at http://kubevirt.io/ ?

If the answer to "how do you run VMs" is "Kubernetes does it" then its about https://kubevirt.io/

We are even using Docker Hub to store and distribute VM images...

https://github.com/kubevirt/kubevirt/blob/main/containerimag...

Docker Desktop runs dockerd in a Linux VM with Apple's hypervisor framework. You can also run containers in a Linux VM with Parallels or VMware Fusion hypervisors. But you can't run VMs inside those VMs as it stands today. This works fine on Intel Macs which means you can't experiment and use KVM - one of the killer features of Linux and things like https://kubevirt.io/

Proxmox VE can run VMs and LXC containers (see my comment below on LXC). Kubernetes can run OCI containers, but there's also KubeVirt for running VMs.

Another option would be something like KubeVirt but that is a different use case where you are actually running a VM in a container for hard-to-containerize workloads.