karpenter-provider-aws VS skaffold

If you’re not familiar with Karpenter — watch my quick intro. But in a nutshell, Karpenter is a better node autoscaler for Kubernetes (say goodbye to wasted compute resources). It is open-source and built by the AWS team. Qovery is an Internal Developer Platform I’m a co-founder) that we’ll use to spin up our EKS cluster with Karpenter.

Karpenter

Here are a few reference links about the previous services and tools: What is Amazon EKS? Cluster Mesh Karpenter

Autoscaling over EKS can be accomplished using either the cluster-autoscaler project or Karpenter. If you want to use Spot instances, consider using Karpenter, as it has better integrations with AWS for optimizing spot pricing and availability, minimizing interruptions, and falling back to on-demand nodes if no spot instances are available.

Kubernetes has its own learning curve, but when tools like Karpenter exist it's kinda hard to beat for "auto-scaled compute" that is vendor agnostic. We leverage Karpenter for burst in our vSphere environment as well as our EC2 environment. Karpenter is invoking roughly the same Terraform code in both cases, just using different modules for the particular virtualization. Say we want to go to Azure and GCP -- we add an Azure and GCP module to the same Terraform codebase, and not much else needs to change from the "scale up / scale down" perspective.

Also https://github.com/aws/karpenter/issues/331

Why don't you stick to the KubernetesPodOperator though? I fail to see a benefit in using the ECS operator considering you're already running Airflow in EKS. You can look into something like karpenter to manage your nodes.

A nice middle ground is using a tool like Google's Skaffold, which provides "Bazel-like" capabilities for composing Docker images and tagging them based on a number of strategies, including file manifests. In my case, I also use build args to explicitly set versions of external dependencies.

While I am in a Typescript environment with this setup at the moment, my personal experience that Skaffold with Docker has a lighter implementation and maintenance overhead than Bazel. (You also get the added benefit of easy deployment and automatic rebuilds.)

I quite liked using Bazel in a small Golang monorepo, but I ran into pain when trying to do things like include third-party pre-compiled binaries in the Docker builds, because of the unusual build rules convention. The advantage of Skaffold is it provides a thin build/tag/deploy/verify layer over Docker and other container types. Might be worth a look!

Kudos to the Google team building it! https://skaffold.dev

To get a similar experience of preevy up, first we’ll need to split the build and deploy using process or alternatively employ tools that orchestrate build-tag-push-update-sync flow like Skaffold/Tilt.

We will be using docker and microk8s from Canonical. For running our software during development, we will be using skaffold which is a great tool developed by Google.

To add more context, if you are developing containers in a local dev environment, the minimum you should have is the Google Cloud SDK and Skaffold. The SDK will allow you to programmatically interact with Googleapis e.g. auth, services, resources. Skaffold will allow you to build and deploy to the cloud similar to working with a local dev environment.

I have used both Skaffold and Devspace. I prefer the latter.

I wonder if it has some overlap with https://skaffold.dev/.

K3d and Skaffold for local development

skaffold.dev - build in k8s - no more asking for the database password. All the plumbing to the backend is just done so it's easier for them to test and demo any branch