karpenter-provider-aws
Portainer
karpenter-provider-aws | Portainer | |
---|---|---|
47 | 337 | |
5,902 | 29,024 | |
3.1% | 1.8% | |
9.9 | 9.8 | |
4 days ago | about 14 hours ago | |
Go | TypeScript | |
Apache License 2.0 | zlib License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
karpenter-provider-aws
- Karpenter
-
Stress testing Karpenter with EKS and Qovery
If you’re not familiar with Karpenter — watch my quick intro. But in a nutshell, Karpenter is a better node autoscaler for Kubernetes (say goodbye to wasted compute resources). It is open-source and built by the AWS team. Qovery is an Internal Developer Platform I’m a co-founder) that we’ll use to spin up our EKS cluster with Karpenter.
- Tortoise: Shell-Shockingly-Good Kubernetes Autoscaling
-
Five tools to add to your K8s cluster
Karpenter
-
Architecting for Resilience: Crafting Opinionated EKS Clusters with Karpenter & Cilium Cluster Mesh — Part 1
Here are a few reference links about the previous services and tools: What is Amazon EKS? Cluster Mesh Karpenter
- Scaling with Karpenter and Empty Pod(A.k.a Overprovisioning)
-
Reducing Cloud Costs on Kubernetes Dev Envs
Autoscaling over EKS can be accomplished using either the cluster-autoscaler project or Karpenter. If you want to use Spot instances, consider using Karpenter, as it has better integrations with AWS for optimizing spot pricing and availability, minimizing interruptions, and falling back to on-demand nodes if no spot instances are available.
-
Help required
Kubernetes has its own learning curve, but when tools like Karpenter exist it's kinda hard to beat for "auto-scaled compute" that is vendor agnostic. We leverage Karpenter for burst in our vSphere environment as well as our EC2 environment. Karpenter is invoking roughly the same Terraform code in both cases, just using different modules for the particular virtualization. Say we want to go to Azure and GCP -- we add an Azure and GCP module to the same Terraform codebase, and not much else needs to change from the "scale up / scale down" perspective.
-
Workload Operator. What do you think?
Also https://github.com/aws/karpenter/issues/331
-
Running Airflow task intensive Dags on Fargate.
Why don't you stick to the KubernetesPodOperator though? I fail to see a benefit in using the ECS operator considering you're already running Airflow in EKS. You can look into something like karpenter to manage your nodes.
Portainer
-
Homelab Adventures: Crafting a Personal Tech Playground
Portainer
-
Runtipi: Docker-Based Home Server Management
> Any tips on the minimum hardware or VPS's needed to get a small swarm cluster setup?
From my testing, Docker Swarm is very lightweight, uses less memory than both Hashicorp Nomad and lightweight Kubernetes distros (like K3s). Most of the resource requirements will depend on what containers you actually want to run on the nodes.
You might build a cluster from a bunch of Raspberry Pis, some old OptiPlex boxes or laptops, or whatever you have laying around and it's mostly going to be okay. On a practical level, anything with 1-2 CPU cores and 4 GB of RAM will be okay for running any actually useful software, like a web server/reverse proxy, some databases (PostgreSQL/MySQL/MariaDB), as well as either something for a back end or some pre-packaged software, like Nextcloud.
So, even 5$/month VPSes are more than suitable, even from some of the more cheap hosts like Hetzner or Contabo (though the latter has a bad rep for limited/no support).
That said, you might also want to look at something like Portainer for a nice web based UI, for administering the cluster more easily, it really helps with discoverability and also gives you redeploy web hooks, to make CI easier: https://www.portainer.io/ (works for both Docker Swarm as well as Kubernetes, except the Kubernetes ingress control was a little bit clunky with Traefik instead of Nginx)
- Cómo instalar Docker CLI en Windows sin Docker Desktop y no morir en el intento
-
Setup Portainer for Server App
In this section, we will add Portainer to help us in managing our Docker containers. You can find more details about it here. To integrate Portainer into our EC2 project, we can follow these steps:
-
Old documentation url on Github issues gives ERR_TOO_MANY_REDIRECTS.
Git issues pointing to: https://docs.portainer.io/v/ce-2.9/start/install/agent/swarm/linux gives a ERR_TOO_MANY_REDIRECTS.
-
Docker CI/CD with multiple docker-compose files.
I am currently running Portainer, but webhooks (GitOps) appear to be broken ( [2.19.0] GitOps Updates not automatically polling from git · Issue #10309 · portainer/portainer · GitHub ) and so I cannot send webhook to redeploy a stack. So, looking for alternatives. Using this as a good excuse to learn more about docker and CI/CD etc.
-
Ask HN: How do you manage your “family data warehouse”?
A Synology NAS running Portainer (https://www.portainer.io/) running Paperless NGX (https://github.com/paperless-ngx/paperless-ngx)
This works better than I can possibly tell you.
I have an Epson WorkForce ES-580W that I bought when my mother passed away to bulk scan documents and it scans everything, double-sided if required, multi-page PDFs if required, at very high speed and uploads everything to OneDrive, at which point I drag and drop everything into Paperless.
I could, thinking about it, have the scanner email stuff to Paperless. Might investigate that today.
Paperless will OCR it and make it all searchable. This setup is amazing, I love living in the future.
-
Bare-Metal Kubernetes, Part I: Talos on Hetzner
> I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for < 100 node cluster, docker swarm should suffice. Easier to setup, maintain and upgrade.
Personally, I'd also consider throwing Portainer in there, which gives you both a nice way to interact with the cluster, as well as things like webhooks: https://www.portainer.io/
With something like Apache, Nginx, Caddy or something else acting as your "ingress" (taking care of TLS, reverse proxy, headers, rate limits, sometimes mTLS etc.) it's a surprisingly simple setup, at least for simple architectures.
-
What are some of your fav panels and why?
casaos it just makes things like backups, offsite syncing and many other nas related things so much easier to manage. And gives you a proper nas like experience similar to that in which you'd fine on companies like tnas or synology. I actually also use it as a replacement for portainer when i don't need the more advanced features it offers
-
Kubernetes Exposed: One YAML Away from Disaster
> I moved to docker swarm and love it. It's so much easier, straight forward, automatic ingress network and failover were all working out of the box. I'll stay with swarm for now.
I've had decent luck in the past with the K3s distribution, which is a bit cut down Kubernetes: https://k3s.io/
It also integrates nicely with Portainer (aside from occasional Traefik ingress weirdness sometimes), which I already use for Swarm and would suggest to anyone that wants a nice web based UI: https://www.portainer.io/
Others might also mention K0s, MicroK8s or others - there's lots of options there. But even so, I still run Docker Swarm for most of my private stuff as well and it's a breeze.
For my needs, it has just the right amount of abstractions: stacks with services that use networks and can have some storage in the form of volumes or bind mounts. Configuration in the form of environment variables and/or mounted files (or secrets), some deployment constraints and dependencies sometimes, some health checks and restart policies, as well as resource limits.
If I need a mail server, then I just have a container that binds to the ports (even low port numbers) that I need and configure it. If I need a web server, then I can just run Apache/Nginx/Caddy and use more or less 1:1 configuration files that I'd use when setting up either outside of containers, but with the added benefit of being able to refer to other apps by their service names (or aliases, if they have underscores in the names, which sometimes isn't liked).
At a certain scale, it's dead simple to use - no need for PVs and PVCs, no need for Ingress and Service abstractions, or lots and lots of templating that Helm charts would have (although those are nice in other ways).
What are some alternatives?
keda - KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
Yacht - A web interface for managing docker containers with an emphasis on templating to provide 1 click deployments. Think of it like a decentralized app store for servers that anyone can make packages for.
autoscaler - Autoscaling components for Kubernetes
swarmpit - Lightweight mobile-friendly Docker Swarm management UI
bedrock - Automation for Production Kubernetes Clusters with a GitOps Workflow
podman - Podman: A tool for managing OCI containers and pods.
karpenterwebsite
OpenMediaVault - openmediavault is the next generation network attached storage (NAS) solution based on Debian Linux. Thanks to the modular design of the framework it can be enhanced via plugins. openmediavault is primarily designed to be used in home environments or small home offices.
dapr - Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.
CasaOS - CasaOS - A simple, easy-to-use, elegant open-source Personal Cloud system.
camel-k - Apache Camel K is a lightweight integration platform, born on Kubernetes, with serverless superpowers
podman-compose - a script to run docker-compose.yml using podman