apt2ostree VS singularity

Hm yes now I remember that point about how the data is anonymous Python objects that you can pass around to functions.

Are there any open source examples? I looked around the github account, but I mostly remember this tool

https://github.com/stb-tester/apt2ostree

I'd be interested in seeing the Python config and Ninja output, to see how it works. Right now it looks to me like the dependencies are more implicit than explicit, e.g. with your copen example

---

The system I ended up with is more like Bazel, but it's not building containers, so it's a slightly different problem. But I'm interested in building containers incrementally without 'docker build'.

I like the apt lockfile idea definitely ... However I also have a bunch of other blobs and tarballs, that I might not want to check into git. I guess you just put those in OSTree?

Our config looks like this

https://github.com/oilshell/oil/blob/master/core/NINJA_subgr...

And all the code is in build/ninja* of the same repo

However, you can get pretty close yourself with a tool like this https://github.com/stb-tester/apt2ostree

With apt2ostree[1] we use lockfiles to allow us to version control the exact versions that were used to build a container. This makes updating the versions explicit and controlled, and building the containers functionally reproducible - albeit not byte-for-byte reproducible.

[1]: https://github.com/stb-tester/apt2ostree#lockfiles

If you have time to test things, you can try to use ostree to manage a Debian installation. This is what Silverblue uses. Their is already a tool to create APT-based ostree images.

On the subject of reproducible debian-based environments I wrote apt2ostree[1]. It applies the cargo/npm lockfile idea to debian rootfs images. From a list of packages we perform dependency resolution and generate a "lockfile" that contains the complete list of all packages, their versions and their SHAs. You can commit this lockfile to git.

You can then install Debian or Ubuntu into a chroot just based on this lockfile and end up with a functionally reproducible result. It won't be completely byte identical as your SSH keys, machine-id, etc. will be different between installations, but you'll always end up with the same packages and package versions installed for a given lockfile.

This has saved us on a few occasions where an apt upgrade had broken the workflow of some of our customers. We could see exactly which package versions changed in git history and roll-back the problematic package before working on fixing it properly. This is vastly better than the traditional `RUN apt-get install -y blah blah` you see in `Dockerfile`s.

IMO it's also more convenient than debootstrap as you don't need to worry about gpg keys, etc. when building the image. Dependency resolution and gpg key stuff is done at lockfile generation time, so the installation process can be much simpler. In theory it could be made such that only dpkg is required to do the install, rather than the whole of apt, but that's by-the-by.

apt2ostree itself is probably not interesting to most people as it depends on ostree and ninja but I think the lockfile concept as applied to debian repos could be of much broader interest.

[1]: https://github.com/stb-tester/apt2ostree#lockfiles

[2]: https://ostreedev.github.io/ostree/

rkt (and many other container solutions) was introduced after docker was released and became popular... they even mentioned docker's shortcomings as a motivation for the project creation [0]. It had all the same problems as other replacement software: there were plenty of bugs and missing features, documentation was limited, and there are no community to help you (the announcement explicitly mentions "prototype quality release"). None of those would be fatal if it was significantly better than docker, but it was not -- it was basically the same functionality. So almost no one made the switch. It is closed now [1]

And why "rkt"? There were much better alternative container runtimes. For example Sylabs Singularity [2] -- container-as-a-file, instant mounting, etc... I wish more people knew about it.

[0] https://web.archive.org/web/20141201181834/https://coreos.co...

[1] https://github.com/rkt/rkt#warning-end-of-project-warning

[2] https://github.com/sylabs/singularity#singularityce

Docker (and the current generation of OCI runtimes) is rather shitty at host OS isolation too. It may have started well, but over time more capabilities were added for the convenience of developers and at the cost of maintainers. I would personally love if the ecosystem were to shift to more isolated workloads with better HPC support. Singualrity looks promising but still maintains OCI compatibility.