init-snapshot VS image-spec

At CodeSandbox we use Firecracker to run our VMs (more info here: https://codesandbox.io/blog/how-we-clone-a-running-vm-in-2-s...).

To answer the questions:

> what version of the kernel do you use (the github page says 5.10 but isn't that quite old?)

Right, they have tested with 5.10, but it also works with higher kernel versions. Our host currently runs 5.19 and we're planning to upgrade to 6.1 soon. The guest runs 5.15.63, we use a config very similar to the recommended config by FC team (it's in the FC repo). It's important to mention that we had to disable async pagefaulting (a KVM feature) with more modern kernel versions, as VMs could get stuck waiting for an PF resolve.

> What do you use to build the 'micro' images

We created a CLI that creates a rootfs from a Docker image. It pulls the image, creates a container and then extracts the fs from it to an ext4 disk. For the init, we forked the open sourced init from the Fly team (https://github.com/superfly/init-snapshot) and changed/added some functionality.

> How do you keep timesync of you're not using a timesync daemon?

IIRC we expose the time as a PTP device (handled by kvm) and run phc2sys to sync the time in an interval. Firecracker has some documentation on this, where it recommends chrony. It can also be done with vsock, but it would be more manual.

> Handle kernel and app logs without adding an log daemon, and same through vsocks, etc?

The init forwards stdout/stderr of the command it runs to its own stdout, which Firecracker then logs out by itself. A supervisor reads these and writes the logs to files.

Unless they’ve changed things, there is no containerization within the VM a la kata. They run their own custom init inside the VM and use it to start the entry point. https://github.com/superfly/init-snapshot is the source.

Jerome wrote our init in Rust, and, after being cajoled by Josh Triplett, [we released the code (https://github.com/superfly/init-snapshot), which you can go read.

> Fly.io transforms container images into fleets of micro-VMs running around the world on our hardware.

Oh boy!

> None of us have ever worked for Google, let alone as SREs. So we’re going out on a limb

Oh.... boy.

> We spent some time scaling it with Thanos, and Thanos was a lot, as far as ops hassle goes.

You know, they have these companies now, that will collect your metrics for you, so that you don't have to deal with ops hassle.

Holy shit. I see they even wrote their own init... in Rust. Yes, the thing that is normally a shell script, is now a compiled program in a new language, that mostly just runs mkdir(), mount() and ethtool(). (https://github.com/superfly/init-snapshot/blob/public/src/bi...)

A buildpack is a software, designed to transform application source code into executable (OCI) images that can run on a variety of cloud platforms. At its core, a buildpack is a directory that includes a specific file named buildpack.toml. This file contains metadata and configuration details that dictate how the buildpack should behave. Buildpacks in simple terms, is a set of standards defining how the different steps that are required to build a compliant container image can be automated. Using those standards, there are projects that have been built round enabling that using an CLI or an API. The most common way of doing that is through the Cloud Native Buildpacks' Pack project. Pack is a CLI command that can run in the same system the developers are using to actually go through creating a Dockerfile.

Eventually, once zstd support gets fully supported, and tiny gzip compression windows are not a limitation, then compressing a full layer would almost certainly have a better ratio over several smaller layers

https://github.com/opencontainers/image-spec/issues/803

Please note that label-schema has been superseded by https://github.com/opencontainers/image-spec/blob/main/annotations.md<^

GitHub Container Registry stores container images within your organization or personal account, and allows you to associate an image with a repository. It currently supports both the Docker Image Manifest V2, Schema 2 and Open Container Initiative (OCI) specifications.

We build all services as containerized workloads, i.e., OCI images - sometimes called Docker images. We deploy these to the Kubernetes product offered by the cloud vendor. Whenever we need some capability, containers are the answer. This insulates our applications from the vendor. In principle, we could switch providers as long as Kubernetes is available.

Build images with anything that makes OCI compliant images, push, and profit.

it's funny that you mention this because it is actually the thing that is next on my agenda for the image, as you can probably see already I bake in OCI image annotations in our image, which is great for including some core pieces of meta data. In addition to this though I will soon be including custom labels for Base64 encoded YAMLs for Kubernetes deployments using this image. I will look at including helm configuration as well. Then it should be just as easy as: $ docker pull registry.gitlab.com/crafty-controller/crafty-4:latest $ docker image inspect registry.gitlab.com/crafty-controller/crafty-4:latest | jq -r ".[].Config.Labels.\"org.arcadiatech.crafty.k8s.deployment\"" | base64 -d | kubectl apply -f -

They still don't do anything really of substance, they're just gateways to their vendor's world - booking systems, payment systems, etc. You learn those as you go along. Yes, as a potential employee, you need to be able to tick those boxes on your CV, but if you understand the underlying technology, it's mostly a matter of booking your own AWS or Azure server for $5-10 a month for a few weeks, and fooling around. (Docker is a bit different in the sense that they were the first to popularize today's de-facto container image standard, the "Docker container", which has since been accepted as a proper standard and renamed to "OCI image format"; but at the end of the day, at this point in time, Docker in itself is still just a company out for the money, and the multi-GB installation of their product can, for the essential functionality part, be replaced by a few hundred lines of Bash code. The cool boys today don't use Docker, they use [Podman(https://podman.io/), which is essentially a much more lightweight drop-in replacement ;-) )