buildkit VS buildah

The IDs associated with each layer were displayed in the build output. Unfortunately, the switch to BuildKit in recent Docker versions prevents the convenient use of intermediate layers because they're no longer stored after the build completes.

Better caching: Railpack interfaces directly with BuildKit to control the layers and filesystem, resulting in more cache hits (with sharable caches across environments)

BuildKit Faster Docker Builds Build speed + cache control = win.

Windows is is very similar, the differences are two the layer tarballs.

The file system appears in a Files sub-directory as there is a Hives sub-directory for containing the Windows Registry.

The other difference is there are two extra PAX headers within the tarball, MSWINDOWS.fileattr which is "32" for a regular file, and "16" for a directory and MSWINDOWS.rawsd which is a special encoding of the security descriptor, which you can think of it as the owner, group and permissions associated with the file (which their standard values can be seen from buildkit here: https://github.com/moby/buildkit/blob/22156ab20bcaea1a1466d2...)

I haven't looked into how to handle the Windows Registry aspect as in my exploration I was focused on simply adding a pre-built executable so I didn't need any registry entries created.

The other fun gotcha is to ensure the ENV section contain PATH set to c:\\Windows\\System32;c:\\Windows otherwise you would be unlikely to be able to run any Windows executable.

BuildKit GitHub Repository

I remember when container builds were slow and not really efficient, and were usually a bottleneck of our CI/CD pipelines. That is until I discovered BuildKit and my life changed. BuildKit is the next-generation builder engine for Docker, but it can also be used independently.

BuildKit is the engine behind Docker's build system. It's a fascinating look into how Docker images are constructed, with features like:

create should be provided with a daemon configuration file through the --buildkitd-config flag (if not, it defaults to the buildkitd.default.toml file contained in the config directory of buildx). You can find an example of a complete configuration file in buildkit official documentation on GitHub.

By default, Kamal uses the docker-container driver to build images which, in turn, uses the BuildKit toolkit internally. While Kamal sets up registry caching correctly, caching still fails in the end because the BuildKit process is isolated from our GitHub Action runtime process. To connect the two, we need to expose the GitHub runtime to the workflow. Luckily, there is a GitHub Action ready just for this so all that is needed is adding the action to the workflow file. We put it right after setting up Docker Buildx:

For the package management, it depends on the package manager, but most have some mechanism for installing into a root other than the currently running system.

Even without explicit support in the pacakage manager, you could also roll your own solution by running the package manager in a chroot environment, which would then need to be seeded with the package manager's own dependencies, of course (and use user-mode qemu to run non-native executables in the case of cross-architecture builds).

Whether this yields a minimal container when pointed at a repository intended to be used to deploy a full OS is another question, but using a package manager to build a root filesystem offline isn't hard to pull off.

As for how to do this in the context of building an OCI container, tools like Buildah[1] exist to support container workflows beyond the conventional Dockerfile approach, providing easy-to-use command line tools to create containers, work with layers, mount and unmount container filesystems, etc.

[1] https://github.com/containers/buildah/blob/main/README.md

Yes, since 2021 they also support the same syntax natively: https://github.com/containers/buildah/pull/3133

Buildah is another tool that can build OCI container images without requiring root privileges:

I avoid dockerfiles and prefer using buildah for building containers. Since they're all using the same specification, it doesn't matter what runtime is then used to run them: it can be docker, podman, k8s, whatever.

Here's the official example of building a lighttpd container:

https://github.com/containers/buildah/blob/92015b7f4301d7eb8...

You can eschew bash and call these commands however you want — from a python script, or Go, or even assembly.

If $PROGRAMMING_LANGUAGE = go, you might be looking for https://github.com/containers/storage which can create layers, images, and so on. I think `Store` is the main entry: https://pkg.go.dev/github.com/containers/storage#Store

Buildah uses it: https://github.com/containers/buildah/blob/main/go.mod#L27C2...

Like wayland, podman was one of those things that took a lot of "I'll give it another shot? Nope, not ready" before I could switch.

Lately the only real incompatibility I run in to with podman is that the handling of `RUN --mount=type=secret` in a Containerfile/Dockerfile is a bit broken (https://github.com/containers/buildah/issues/5282).

I wish we would rather get rid of Dockerfile in favor of something like buildah does:

https://github.com/containers/buildah/blob/main/examples/lig...

Since Dockerfile is a rather limited and (IMHO) poorly executed re-implementation of a shell script, why not rather use shell directly? Not even bash with coreutils is necessary: even posix sh with busybox can do far more than Dockerfile, and you can use something else (like Python) and take it very far indeed.

For the task of building the graph image, my first idea was to rely on buildah. In fact, our design was already heavily relying on containers/image for all things regarding copying images from one registry to the other, or from one registry to an archive. The obvious choice was to use the same suite of modules in order to keep dependencies to a minimum.

I appreciate that podman can run daemonless, but I've gotten tired of waiting for them to implement heredoc support and have continued to use docker.