kata-containers VS cloud-hypervisor

> I actually wonder how much "overhead" a VM actually has. i.e. a linux kernel that doesn't do anything (say perhaps just boots to an init that mounts proc and every n seconds read in/prints out /proc/meminfo) how much memory would the kernel actually be using?

There's already some memory sharing available using DAX in Kata Containers at least: https://github.com/kata-containers/kata-containers/blob/main...

Seems like Fly.io Machines are trying reimplement Kata Containers with the Firecracker backend [0].

Kata has a guest image and guest agent to run multiple isolated containers [1].

[0] https://katacontainers.io/

[1] https://github.com/kata-containers/kata-containers/blob/main...

> Last time I looked (a few months ago), the documentation was pretty sparse or outdated.

It still is, though it works somewhat seamlessly when installing with https://github.com/kata-containers/kata-containers/blob/main...

Though only one of the hypervisors works well.

Better to use a secure VM, can even get container-like VMs with kata-containers

As I understand,Kata Containers

Kubernetes using Kata containers as a containerd backend

https://github.com/kata-containers/kata-containers/blob/main...

My home k8s cluster is now "locked down" using micro-vms (kata-containers[0]), pod level firewalling (cilium[1]), permission-limited container users, and mostly immutable environments. Given how quickly I rolled this out; the tools to enhance cluster environment security seem more accessible now than my previous research a few years ago.

I know it's not exactly a production setup, but I really do feel that it's the most secure runtime environment I've ever had accessible at home. Probably more so than my desktops, which you could argue undermines most of my effort, but I like to think I'm pretty careful.

In the beginning I was very skeptical, but being able to just build a docker/OCI image and then manage its relationships with other services with "one pane of glass" that I can commit to git is so much simpler to me than my previous workflows. My previous setup involved messing with a bunch of tools like packer, cloud-init, terraform, ansible, libvirt, whatever firewall frontend was on the OS, and occasionally sshing in for anything not covered.

[0] https://github.com/kata-containers/kata-containers

If the calls first pass through a memory safe language as what gvisor does, isn’t the attack surface greatly reduced?

It does seem however that Firecracker + GPU support (or https://github.com/cloud-hypervisor/cloud-hypervisor) is most promising though.

There is no mention of cloud-hypervisor[1] (also in the rust-vmm ecosystem) in the article. It has the memory reclamation feature they require. It also support VFIO and virtiofs.

[1] <https://github.com/cloud-hypervisor/cloud-hypervisor>

https://github.com/tandasat/Hypervisor-101-in-Rust is there to help

https://github.com/cloud-hypervisor/cloud-hypervisor isn't educational necessarily but is one of the most technically progressive fastest developing highest funded vm projects ever, and there are oodles of tech talks on it. I am not qualified to make any specific recommendations, but there's tons of stuff here.

> The goal of the Cloud Hypervisor project differs from the aforementioned projects in that it aims to be a general purpose VMM for Cloud Workloads and not limited to container/serverless or client workloads.

Firecracker is such a great piece of technology. I'm amazed that AWS actually open-sourced it. All kudos to them. We're using Firecracker at our company to allow API companies build interactive demos like this one we built for Prisma [1].

[0] https://github.com/cloud-hypervisor/cloud-hypervisor

[1] https://playground.prisma.io

Relatively new project 'Cloud Hypervisor' https://github.com/cloud-hypervisor/cloud-hypervisor seems to launch images faster.

The Chrome OS hypervisor was then evolved/forked into Firecracker and Intel's Cloud Hypervisor, with the latter supporting both Linux and Windows. Perhaps Cloud Hypervisor would serve as a good backbone for sandboxing, with its Rust implementation and focus on security?

GitHub - cloud-hypervisor/cloud-hypervisor: A Virtual Machine Monitor for modern Cloud workloads. Features include CPU, memory and device hotplug, support for running Windows and Linux guests, device offload with vhost-user and a minimal compact footprint. Written in Rust with a strong focus on security.

Did you guys think about live migrations? https://github.com/cloud-hypervisor/cloud-hypervisor seems to support it and it shares a good amount of code with firecracker.