skopeo VS Bazel

I decided to go searching for an alternative means to pull a docker image. In my search I discovered Skopeo, an alternative method to download Docker images that proved to be surprisingly effective. It not only downloaded the image faster, it also allowed me to save my image in a tar file, which means you can pull an image on one system and share that image to another system, loading it easily to docker instance on that system. This can be very beneficial if you have multiple systems and don't want to download an image multiple times.

But I'd suggest looking into if it's solved by other tools already, like regclient/regclient and their regsync features or something like containers/skopeo.

Have a use case where we have a CLI (built with cobra) for our dev teams which can execute common tasks. One of those tasks we want to implement is to copy docker images from the internet to our internal registry. A tool such as skopeo can do this and much more. Instead of essentially re-writing the functionality directly into our CLI we'd like to embed it. This would also negate the need for the dev teams to manage multiple CLI tools.

Self hoisting here, I put this together to make it easier to generate single (extra) layer docker images without needing a docker agent, capabilities, chroot, etc: https://github.com/andrewbaxter/dinker

Caveat: it doesn't work on Fly.io. They seem to be having some issue with OCI manifests: https://github.com/containers/skopeo/issues/1881 . They're also having issues with new docker versions pushing from CI: https://community.fly.io/t/deploying-to-fly-via-github-actio... ... the timing of this post seems weird.

FWIW the article says

> create a Docker image, also known as an OCI image

I don't think this is quite right. From my investigation, Docker and OCI images are basically content addressed trees, starting with a root manifest that points to other files and their hashes (root -> images -> layers -> layer configs + files). The OCI manifests and configs are separate to Docker manifests and configs and basically Docker will support both side by side.

skopeo is another tool worth looking into. we've started deploying amd and arm nodes into our k8s clusters, and this tool was incredibly easy to build around for getting multi-arch images into our container registry.

I would use skopeo, the tool is quite handy for working with remote images. https://github.com/containers/skopeo

Using distroless images not only reduces the size of the container image it also reduces the surface attack. The need for container image signing is because even with the distroless images there is a chance of facing some security threats such as receiving a malicious image. We can use cosign or skopeo for container signing and verifying. You can read more about securing containers with Cosign and Distroless Images in this blog.

Look at using tools like skopeo or crane

You could try some commandline tool like skopeo to fetch the image tags regularly and do some shell magic to notify you on any change you want

This is what Podman, an open-source daemonless and rootless container engine, was developed with in mind. Podman runs using the runC container runtime process, directly on the Linux kernel, and launches containers and pods as child processes. In addition, it was developed for the Docker developer, with most commands and syntax seamlessly mirroring Docker's. Buildah, an image builder, and Skopeo, the image utility tool, are both complimentary to Podman as well, and extend the range of operations able to be performed.

Wow, if you curl it, there's a lot of boilerplate code there.

Maybe built using Bazel?

https://bazel.build

Bazel by Google

Luckily a feature to limit the disk cache size is in development: https://github.com/bazelbuild/bazel/issues/5139

This is a problem that Bazel (https://bazel.build) solves in a very convenient way. You can just keep using the paths relative to the repository root, and as long as you properly declare your test needs that file it will access it without problems. Or you can use the runfile libraries to access them too.

When doing research for this lab exercise I looked at both vcpkg and conan. Both are package managers that would automate the installation and configuration of my program with its dependencies. However, when it came to releasing and sharing my program my options were limited. For example, the central public registry for conan packages is conan-center, but these packages are curated and the process is very involved. There was no way conan-center would accept a class project like mine. Alternatively, I could host a conan package on a public Artifactory repository, but accessing the package requires users to add the repository to their conan remote. This already sounded like too many steps to expect regular users to follow - I already haven't setup any conan remotes, there's no way I could expect regular users to know about conan remotes, let alone have conan installed on their system. After discussing with people online and consulting my instructor, I ultimately decided to do a GitHub release. However, in the future I was encouraged to look into using CMake or bazel.

NOTE: I won’t mention SBT and Leiningen here because, with all due respect, they are niche build tools. I also won’t discuss Kobalt for the same reason (besides, it’s no longer actively maintained). Additionally, I won’t touch upon Bazel and Buck in this context, mainly because I’m not very familiar with them. If you have insights or comments about these tools, please feel free to share them in the comments 👇

> None of this solves C's only REAL problem (in my opinion) which is the lack of dependency management.

Bazel solves this really nicely, I know some people have strong opinions on it but I cannot recommend it enough

https://bazel.build/