k8s-config-connector VS cachemachine

If your company is k8s centric and the developers are most comfortable with k8s, you might want to focus more on something specific to k8s such as helm, or even if you don't get into helm you may want to use config connector in your yaml to manage GCP resources in an IaC compliant method. You can manage k8s resources with terraform, but if your developers are currently comfortable working directly with k8s you are going to see significant pushback getting them to add terraform as a middleware. You probably still want to manage your GKE clusters and VPCs with terraform since you can't really use config connector.

Config Connector [1] is also an option in this space for GCP, it supports many GCP resources and thus far our experience with it has been largely positive.

[1] https://cloud.google.com/config-connector/docs/overview

We use it heavily with GCP's Kubernetes Config Connector to provision architecture. It could similarly be used for Cloud Functions, etc. given a repo URL that GCP can access. GitOps + operator pattern is a pretty powerful mechanism to let k8s continuously seek state towards your ideal. https://cloud.google.com/config-connector/docs/overview

For K8s, we were using Cloud Composer to do it for us but wanted more fine control over CI/CD, so we decided to go with Airflow on K8s. That's all hosted in GKE now and deployed using ArgoCD with helm. This also led down the IaC rabbit hole which has been a ton of fun too. We use the GCP ConfigConnector resources for that which is a little challenging at first, but gets a lot easier as time goes on.

Terraform is 90% of cloud IaC. But there are newer Kubernetes Operators like Config Connector that can create cloud specific resources.

It works particularly well with Google Config Connector as then it's all just manifests.

> we found the start times with Kubernetes too slow

Just curious if you could elaborate here? I work with k8s on docker, and we're also going to be spinning up ephemeral containers (and most of the other things you say) with jupyter notebooks. We're all in on k8s, but since you might be ahead of me, just wondering what hurdles you have faced?

Our big problem was fetching containers took too long since we have kitchen sink containers that are like 10 GB (!) each. They seem to spin up pretty fast though if the image is already pulled. I've worked on a service that lives in the k8s cluster to pull images to make sure they are fresh (https://github.com/lsst-sqre/cachemachine) but curious if you are talking about that or the networking?