-
Lean and Mean Docker containers
Slim(toolkit): Don't change anything in your container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
Packer
Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
In my experience docker-slim[0] is the way to go for creating minimal and secure Docker images.
I wasted a lot of time in the past trying to ship with Alpine base images and statically compiling complicated software. All the performance, compatibility, package availability headaches this brings is not worth it when docker-slim does a better job of removing OS from your images while letting you use any base image you want.
Tradeoff is that you give up image layering to some extent and it might take a while to get dead-file-elimination exactly right if your software loads a lot of files dynamically (you can instruct docker-slim to include certain paths and probe your executable during build).
If docker-slim is not your thing, “distroless” base images [1] are also pretty good. You can do your build with the same distro and then in a multi stage docker image copy the artifacts into distroless base images.
[0] https://github.com/docker-slim/docker-slim
[1] https://github.com/GoogleContainerTools/distroless
, but I prefer to not have to make this assumption and use an init system instead.
[1]: https://github.com/Yelp/dumb-init
I would recommend Google Ko if you are packaging Go apps: https://github.com/google/ko
I'm applying security patches, necessary updates and similar during system image creation (VM image - for example AWS AMI - the one later referred in Dockerfile's FROM). Hashicorp's Packer[1] comes in handy. System images are built and later tested in an automated fashion with no human involvement.
Testing phase involves building Docker image from fresh system image, creating container(s) from new Docker image and testing resulting systems, applications and services. If everything goes well, the system image (not Docker image) replaces previously used system image (one without current security patches).
We have somewhat dynamic and frequent Docker images creation. Subsequent builds based on the same system image are consistent and don't cause problems like inability to scale. Docker does not mess with the system prepared by Packer - doesn't run apt, download from 3rd party remote hosts but only issues commands resulting in consistent results.
This way we no longer have issues like inability to scale using new Docker images and humans are rarely bothered outside testing phase issues. No problems with containers though, as no untested stuff is pushed to registries.
[1] https://www.packer.io/
Earthly is great (disclosure: work on it)
But also checkout out IckFiles, an Intercal frontend for moby buildkit:
https://github.com/adamgordonbell/compiling-containers/tree/...
- a final RUN chown -R statement (~750GB)
We reduced the image size and relative Spark cluster considerably by playing around with dependencies in order to stick with plain pip and using COPY --chown.
I also recommend [dive](https://github.com/wagoodman/dive) analyse what contributes to each layer.
I think the OP is confusing the runtime and image format a bit here. At runtime OverlayFS can use metadata-only copy up to describe changed files, but the container image is still defined as a sequence of layers where each layer is a tar file. There's no special handling for metadata-only changes of a file from a parent layer. As the OCI image spec puts it [1]:
> Additions and Modifications are represented the same in the changeset tar archive.
[1]: https://github.com/opencontainers/image-spec/blob/02efb9a75e...