kubernetes-replicator VS docker-compose-stack

I've built a small MutatingAdmissionWebhook controller [0] that handles this, via a pod annotation whose value is a secret with `ca.crt` inside, and it uses the (mostly) de facto standard openssl variables to configure the libraries, so that it works across pretty much everything I've tried it with off the shelf.

I build a bundle (though I may just move to trust-manager [1]) and replicate it into all namespaces with kubernetes-replicator [2], and then I can annotate any pod with

[0] https://github.com/microcumulus/ca-injector

[1] https://github.com/cert-manager/trust-manager

[2] https://github.com/mittwald/kubernetes-replicator

Whichever way you go, I’ve successfully used this to replicate secrets: https://github.com/mittwald/kubernetes-replicator

You're right, it's a little weird. I wrote a short essay about my setup[1] but the tl;dr is that I wanted certificates distributed in the same way every other thing on my machines is distributed.

I wanted my homeprod setup to be as hands off as possible while still allowing easy management. Each physical host is running Alpine. During provisioning I install docker, Tailscale, and manually start a "root" container that runs[2] docker compose and then starts a cron daemon. The compose commands include one or more "stack" files and are generated based on a yaml file listing the stacks for each host. Watchtower runs with a 30 second cycle time to keep everything updated, including the root container. Adding or updating services means committing and pushing a change to the root container repo, then CI builds and pushes a new image. Watchtower picks up the new image and restarts the root container, which re-runs Compose which in turn starts, stops, modifies, etc anything that's changed.

For certificates, I tried a number of different things but ultimately settled on the method I described earlier. The purpose of the container image is to 1) transport the certificates and install them in the right spot and 2) be updatable automatically with Watchtower.

Certificate changes are very similar to the root container, except the git repo self-modifies upon renewals (yes I keep private keys committed to git, it's a homelab, it's really not a big deal).

[1]: https://www.petekeen.net/homeprod-management-with-docker

[2]: https://github.com/peterkeen/docker-compose-stack/blob/main/...

I really like this article just for the straightforwardness of the setup. Pets not cattle should be the home server mantra.

My setup is not quite as simple. I have one homeprod server running Proxmox with a number of single task VMs and LXCs. A task, for my purposes, is a set of one or more related services. So I have an internal proxy VM that also runs my dashboard. I have a media VM that runs the *arrs. I have an LXC that runs Jellyfin (GPU pass through is easier with LXC). A VM running Home Assistant OS. Etcetera.

Most of these VMs are running Docker on top of Alpine and a silly container management scheme I've cooked up[1]. I've found this setup really easy to wrap my head around, vs docker swarm or k8s or what have you. I'm even in the process of stripping dokku out of my stack in favor of this setup.

[1]: https://github.com/peterkeen/docker-compose-stack