cachemachine VS kube-fledged

> we found the start times with Kubernetes too slow

Just curious if you could elaborate here? I work with k8s on docker, and we're also going to be spinning up ephemeral containers (and most of the other things you say) with jupyter notebooks. We're all in on k8s, but since you might be ahead of me, just wondering what hurdles you have faced?

Our big problem was fetching containers took too long since we have kitchen sink containers that are like 10 GB (!) each. They seem to spin up pretty fast though if the image is already pulled. I've worked on a service that lives in the k8s cluster to pull images to make sure they are fresh (https://github.com/lsst-sqre/cachemachine) but curious if you are talking about that or the networking?

Exactly. Part of the appeal to consolidate all of our container images to use Debian-slim is the ability to optimise the caching of layers, both in our container registry but also on our kubernetes cluster’s nodes (which can be done in a consistent manner with kube-fledged[1]).

[1] https://github.com/senthilrch/kube-fledged

If you're pulling big images you could try kube-fledged (it's the simplest option, a CRD that works like a pre-puller for your images), or if you have a big cluster you can try a p2p distributor, like kraken or dragonfly2.

Also there's that project called Nydus that allows starting up big containers way faster. IIRC, starts the container before pulling the whole image, and begins to pull data as needed from the registry.

https://github.com/senthilrch/kube-fledged

https://github.com/dragonflyoss/Dragonfly2

https://github.com/uber/kraken

https://nydus.dev/

kube fledged - pre pull containes in nodes: https://github.com/senthilrch/kube-fledged

kube-fledged is a kubernetes add-on or operator for creating and managing a cache of container images directly on the worker nodes of a kubernetes cluster. It allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pulled). As a result, application pods start almost instantly, since the images need not be pulled from the registry. kube-fledged provides CRUD APIs to manage the lifecycle of the image cache, and supports several configurable parameters in order to customize the functioning as per one’s needs. (URL: https://github.com/senthilrch/kube-fledged)

I found recently this tool kube-fledged that should do what you want..

I recommend you also look at kube-fledged this is more appealing IMHO.