apt2ostree VS world-atlas

Hm yes now I remember that point about how the data is anonymous Python objects that you can pass around to functions.

Are there any open source examples? I looked around the github account, but I mostly remember this tool

https://github.com/stb-tester/apt2ostree

I'd be interested in seeing the Python config and Ninja output, to see how it works. Right now it looks to me like the dependencies are more implicit than explicit, e.g. with your copen example

---

The system I ended up with is more like Bazel, but it's not building containers, so it's a slightly different problem. But I'm interested in building containers incrementally without 'docker build'.

I like the apt lockfile idea definitely ... However I also have a bunch of other blobs and tarballs, that I might not want to check into git. I guess you just put those in OSTree?

Our config looks like this

https://github.com/oilshell/oil/blob/master/core/NINJA_subgr...

And all the code is in build/ninja* of the same repo

However, you can get pretty close yourself with a tool like this https://github.com/stb-tester/apt2ostree

With apt2ostree[1] we use lockfiles to allow us to version control the exact versions that were used to build a container. This makes updating the versions explicit and controlled, and building the containers functionally reproducible - albeit not byte-for-byte reproducible.

[1]: https://github.com/stb-tester/apt2ostree#lockfiles

On the subject of reproducible debian-based environments I wrote apt2ostree[1]. It applies the cargo/npm lockfile idea to debian rootfs images. From a list of packages we perform dependency resolution and generate a "lockfile" that contains the complete list of all packages, their versions and their SHAs. You can commit this lockfile to git.

You can then install Debian or Ubuntu into a chroot just based on this lockfile and end up with a functionally reproducible result. It won't be completely byte identical as your SSH keys, machine-id, etc. will be different between installations, but you'll always end up with the same packages and package versions installed for a given lockfile.

This has saved us on a few occasions where an apt upgrade had broken the workflow of some of our customers. We could see exactly which package versions changed in git history and roll-back the problematic package before working on fixing it properly. This is vastly better than the traditional `RUN apt-get install -y blah blah` you see in `Dockerfile`s.

IMO it's also more convenient than debootstrap as you don't need to worry about gpg keys, etc. when building the image. Dependency resolution and gpg key stuff is done at lockfile generation time, so the installation process can be much simpler. In theory it could be made such that only dpkg is required to do the install, rather than the whole of apt, but that's by-the-by.

apt2ostree itself is probably not interesting to most people as it depends on ostree and ninja but I think the lockfile concept as applied to debian repos could be of much broader interest.

[1]: https://github.com/stb-tester/apt2ostree#lockfiles

[2]: https://ostreedev.github.io/ostree/

It looks like the author gave up on using Make with this commit: https://github.com/topojson/world-atlas/commit/8f92f6eb692d1...

The article has the following near the end:

"To see more real-world examples of makefiles, see my [World Atlas](https://github.com/mbostock/world-atlas) and [U.S. Atlas](https://github.com/topojson/us-atlas) projects, which contain makefiles for generating TopoJSON from Natural Earth, the National Atlas, the Census Bureau, and other sources."

I checked those repositories because the descriptions of the makefiles sound interesting, but I couldn't find the makefiles. Am I looking wrong?