linuxkit VS firecracker-container

Another project that aims to deliver this is Linuxkit (https://github.com/linuxkit/linuxkit). All the components they ship are written in memory safe languages (usually Go) and run as containers under containerd. You can build a custom image very easily, fully defined as a YAML file.

docker-the-company maintained https://github.com/linuxkit/linuxkit when I worked there. I have no idea who maintains it now, but it looks like it is still active (presumably still docker-the-company, since their adopters list [1] lists docker desktop).

[1]: https://github.com/linuxkit/linuxkit/blob/master/ADOPTERS.md

LF-Edge EVE project leverages Linuxkit to create custom OSs for Edge Devices which in turn leverages Containers as Lego Blocks

LinuxKit - Docker

Behind the scenes Docker Desktop for Mac spawns a linuxkit VM with a bit of extra stuff like NFS to enable mounting Mac paths into containers. In the Docker Desktop settings you'll find the current resource assignment for that VM. That is pretty much reserved for docker so that it does not have to compete with MacOS processes for available resources.

That said... you might want to check out linuxkit

The definition of what a unikernel is needs to be narrowed down, a lot of these projects in the space (not all the ones listed above) have material differences that are not clear:

- some run only one language

- some require recompilation

- some essentially swap out libraries, others do something closer to dropping your already mostly static binary in a minimal disk image

- some build pid1 processes, others VMs images

Anyway, here are some additional entries in the space:

- https://ssrg-vt.github.io/hermitux/

- https://github.com/linuxkit/linuxkit (more embedded/minimal VM than unikernel)

- https://nabla-containers.github.io/ (runs on Solo5)

I am going through using Linuxkit to build AMIs for cloud providers now. I wouldn’t necessarily class linuxkit as a universal project because it doesn’t have the hallmark blurring of user and kernel space or kernel-as-a-library but you can customize the kernel so it’s an adjacent idea, and I think it’s the one most likely to be in actual use at non-hyperscalers.

I believe there is growing interest in providing leaner, "trimmed" runtimes for services deployed to the cloud. Today, this is seen largely by specializing the Linux kernel for, for example, container services[0] or in general[1], as much as that is possible (the paper above covers this problem in greater detail). But, Unikernels in themselves are not yet widely adopted. This is the space Unikraft is aiming to enter, providing the ultimate level of specialization for a target application.

It's clear that bigger players, such as Red Hat[2] are interested in the topic of unikernels, and that cloud providers are preparing for this future too [3].

[0]: https://github.com/linuxkit/linuxkit

There is this project, which I have never used, but seems promising. https://github.com/firecracker-microvm/firecracker-container...

How does that compare to firecracker-containerd?

https://github.com/firecracker-microvm/firecracker-container...

This repository enables the use of a container runtime, containerd, to manage Firecracker microVMs. Like traditional containers, Firecracker microVMs offer fast start-up and shut-down and minimal overhead. Unlike traditional containers, however, they can provide an additional layer of isolation via the KVM hypervisor.

I'm really impressed by fly.io, and the candidness with which they share some of their really awesome technology. Being container-first is the next step for PaaS IMO and they are ahead of the pack.

I aim to build a platform like theirs someday (probably not any time soon) but I don't think I'd do any of what they're doing -- it feels unnecessary. Bear with me as I recently learned that they use nomad[0] and some of these suggestions are kubernetes projects but I'd love to hear why the following technologies were decided against (if they were):

- kata-containers[1] (it does the whole container -> VM flow for you, automatically, nemu, firecracker) with multiple VMM options[2]

- linuxkit[3] (let's say you didn't go with kata-containers, this is another container->VM path)

- firecracker-containerd[4] (very minimal keep-your-container-but-run-it-as-a-VM)

- kubevirt[5] (if you just want to actually run VMs, regardless of how you built them)

- Ceph[6] for storage -- make LVM pools and just give them to Ceph, you'll get blocks, distributed filesystems (CephFS), and object gateways (S3/Swift) out of it (in the k8s space Rook manages this)

As an aside to all this, there's also LXD, which supports running "system" (user namespace isolated) containers, VMs (somewhat recent[7][8]), live migration via criu[9], management/migration of underlying filesystems, runs on LVM or zfs[10], it's basically all-in-one, but does fall behind in terms of ecosystem since everyone else is aboard the "cloud native"/"works-with-kubernetes" train.

I've basically how I plan to run a service like fly.io if I ever did -- so maybe my secret is out, but I sure would like to know just how much of this fly.io got built on (if any of it), and/or what was turned down.

[0]: https://news.ycombinator.com/item?id=26745514

[1]: https://github.com/kata-containers/kata-containers

[2]: https://github.com/kata-containers/kata-containers/blob/2fc7...

[3]: https://github.com/linuxkit/linuxkit

[4]: https://github.com/firecracker-microvm/firecracker-container...

[5]: https://github.com/kubevirt/kubevirt

[6]: https://docs.ceph.com/

[7]: https://discuss.linuxcontainers.org/t/running-virtual-machin...

[8]: https://github.com/lxc/lxd/issues/6205

[9]: https://criu.org/Main_Page

[10]: https://linuxcontainers.org/lxd/docs/master/storage