buildkit VS dive

The IDs associated with each layer were displayed in the build output. Unfortunately, the switch to BuildKit in recent Docker versions prevents the convenient use of intermediate layers because they're no longer stored after the build completes.

Better caching: Railpack interfaces directly with BuildKit to control the layers and filesystem, resulting in more cache hits (with sharable caches across environments)

BuildKit Faster Docker Builds Build speed + cache control = win.

Windows is is very similar, the differences are two the layer tarballs.

The file system appears in a Files sub-directory as there is a Hives sub-directory for containing the Windows Registry.

The other difference is there are two extra PAX headers within the tarball, MSWINDOWS.fileattr which is "32" for a regular file, and "16" for a directory and MSWINDOWS.rawsd which is a special encoding of the security descriptor, which you can think of it as the owner, group and permissions associated with the file (which their standard values can be seen from buildkit here: https://github.com/moby/buildkit/blob/22156ab20bcaea1a1466d2...)

I haven't looked into how to handle the Windows Registry aspect as in my exploration I was focused on simply adding a pre-built executable so I didn't need any registry entries created.

The other fun gotcha is to ensure the ENV section contain PATH set to c:\\Windows\\System32;c:\\Windows otherwise you would be unlikely to be able to run any Windows executable.

BuildKit GitHub Repository

I remember when container builds were slow and not really efficient, and were usually a bottleneck of our CI/CD pipelines. That is until I discovered BuildKit and my life changed. BuildKit is the next-generation builder engine for Docker, but it can also be used independently.

BuildKit is the engine behind Docker's build system. It's a fascinating look into how Docker images are constructed, with features like:

create should be provided with a daemon configuration file through the --buildkitd-config flag (if not, it defaults to the buildkitd.default.toml file contained in the config directory of buildx). You can find an example of a complete configuration file in buildkit official documentation on GitHub.

By default, Kamal uses the docker-container driver to build images which, in turn, uses the BuildKit toolkit internally. While Kamal sets up registry caching correctly, caching still fails in the end because the BuildKit process is isolated from our GitHub Action runtime process. To connect the two, we need to expose the GitHub runtime to the workflow. Luckily, there is a GitHub Action ready just for this so all that is needed is adding the action to the workflow file. We put it right after setting up Docker Buildx:

dive (~54k⭐) lets you browse what's in it — file by file, interactively. It answers what is in a layer.

dive on GitHub — Docker image layer explorer

A similar tool to this (includes interactive TUI) is https://github.com/wagoodman/dive

To understand and optimize the IRIS Community Edition image, we used dive, a tool for inspecting Docker image layers and identifying inefficiencies.

But while that doesn't happen, I’d like to share a Dockerfile structure I’ve been working on. Modesty aside, I think the result is pretty solid 😎. In my case, the initial image was around 1GB, and after some tweaks, it dropped to about 161MB (according to Dive)); it’s worth noting this was for a simple app built with [Hono](https://hono.dev/.

You can also inspect the content of image layers using Dive. Dive is a popular open-source community tool for visualizing container images. It provides an interactive terminal interface.

You're bang on, but you can do things with dive (https://github.com/wagoodman/dive) and use chunks of the code in other projects... That's what I've been doing. The license is MIT so it's permissive.

But yes, an API would be ideal. I've wasted far too much time on this.

Docker images can easily grow with outdated dependencies or security flaws. Tools like Dive or docker scout help analyze image layers.

Dive Visualize image bloat