firebuild VS kata-containers

https://github.com/combust-labs/firebuild

Build firecracker vmms from dockerfiles. Learned a ton, never did anything with it but I want believe it influenced a bunch of people to look deeper into firecracker.

A Dockerfile is just a file with a bunch of commands to execute and get a working "computer". https://github.com/combust-labs/firebuild is fairly aged translation of the Dockerfile to a VM rootfs.

Regarding turning Dockerfiles into a MicroVM: https://gruchalski.com/posts/2021-03-23-introducing-firebuil..., on GitHub: https://github.com/combust-labs/firebuild. This could get you started.

Disclaimer: I’m the author.

> See, and in almost all of my use-cases, I really do. I do HPC computing, which is almost always a multi-tenant environment.

Maybe you need firecracker with something along the lines of https://github.com/combust-labs/firebuild?

I was the one who posted that question and I deleted it because I understood I was asking to compare apple to oranges. Firecracker uses KVM under the hood. With regards to the overhead, sure, there's going to be resources needed but firecracker is pretty good at running VMs will low level overhead. We're talking thousands on a single host.

I've done some fair share of evaluating firecracker for https://github.com/combust-labs/firebuild and the need to provision is red herring. firebuild can run a VM directly from a Dockerfile and Docker image. Fly.io team does something similar. It's basically a fully functional app out of the container within a matter of milliseconds.

What would be nice is to see a direct comparison between your solution and firecracker.

I am working on something like this: https://github.com/combust-labs/firebuild.

From a Dockerfile, it's not as simple without creating an image first.

I bet the first thing you think that it is a bug. There is an issue on GitHub where someone thought the same. The fact is that Kata containers are different and there are Limitations. The first I noticed too, that there is no way to share process or network namespaces between Docker containers. The fact that you cannot use the process namespace or network namespace of the host is easily understandable because we have a VM and not just a host kernel isolating our processes.

> I actually wonder how much "overhead" a VM actually has. i.e. a linux kernel that doesn't do anything (say perhaps just boots to an init that mounts proc and every n seconds read in/prints out /proc/meminfo) how much memory would the kernel actually be using?

There's already some memory sharing available using DAX in Kata Containers at least: https://github.com/kata-containers/kata-containers/blob/main...

Seems like Fly.io Machines are trying reimplement Kata Containers with the Firecracker backend [0].

Kata has a guest image and guest agent to run multiple isolated containers [1].

[0] https://katacontainers.io/

[1] https://github.com/kata-containers/kata-containers/blob/main...

> Last time I looked (a few months ago), the documentation was pretty sparse or outdated.

It still is, though it works somewhat seamlessly when installing with https://github.com/kata-containers/kata-containers/blob/main...

Though only one of the hypervisors works well.

Better to use a secure VM, can even get container-like VMs with kata-containers

As I understand,Kata Containers

Kubernetes using Kata containers as a containerd backend

https://github.com/kata-containers/kata-containers/blob/main...

My home k8s cluster is now "locked down" using micro-vms (kata-containers[0]), pod level firewalling (cilium[1]), permission-limited container users, and mostly immutable environments. Given how quickly I rolled this out; the tools to enhance cluster environment security seem more accessible now than my previous research a few years ago.

I know it's not exactly a production setup, but I really do feel that it's the most secure runtime environment I've ever had accessible at home. Probably more so than my desktops, which you could argue undermines most of my effort, but I like to think I'm pretty careful.

In the beginning I was very skeptical, but being able to just build a docker/OCI image and then manage its relationships with other services with "one pane of glass" that I can commit to git is so much simpler to me than my previous workflows. My previous setup involved messing with a bunch of tools like packer, cloud-init, terraform, ansible, libvirt, whatever firewall frontend was on the OS, and occasionally sshing in for anything not covered.

[0] https://github.com/kata-containers/kata-containers