Kubernetes-Volume-Autoscaler
autoscaler
Kubernetes-Volume-Autoscaler | autoscaler | |
---|---|---|
16 | 89 | |
245 | 7,622 | |
1.2% | 0.7% | |
7.1 | 9.7 | |
7 months ago | 5 days ago | |
Python | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Kubernetes-Volume-Autoscaler
- Toyota blames factory shutdown in Japan on ‘insufficient disk space’
-
[AWS] EKS vs Self managed HA k3s running on 1x2 ec2 machines, for medium production workload
Additionally if you don't know, Kubernetes freshly setup, especially AWS's EKS is largely useless after you first set it up. You need to then install roughly a dozen other services into it to make it "do all the magic automatically". Services such as aws-ebs-csi-driver, (optional) aws-efs-csi-driver, (optional) aws-fsx-csi-driver, aws-load-balancer-controller, (optional) aws-node-termination-handler, cluster-autoscaler, (optional) external-dns, logs cascading engine (eg: fluentd-elasticsearch / fluent-bit-elasticsearch / datadog), grafana, prometheus, your ingress controller of choice (I prefer and recommend ingress-nginx), and the Kubernetes Volume Autoscaler to auto-scale up EBS volumes. (shameless plug: I wrote the volume-autoscaler)
-
Monitoring many cluster k8s
Shameless Plug: Here's one of my dashboards I made for Ingress-Nginx, which is my recommended border router/gateway into all the services. It adds deep robust metrics and configurability, and if you've got years of experience with Nginx also, it allows you rich complex customization via nginx's configuration structure via kubernetes annotations. Besides that I have open-source helm charts which are easy to use, boilerplates showing how to use them, a volume autoscaler to automatically resize your disks as they get full, and a blog where I share various of my experience which is a companion blog to my upcoming book of the same name. Hope this helps! Feel free to ask if you have any further questions.
-
QUESTION: What is the best way to learn kubernetes?
Do not waste your timesetting up your own Kubernetes cluster; use any cloud provider's fully managed Kubernetes cluster, and then learn how to configure everything on it to do everything you want. Typically, there are anywhere between 10-30 foundational services you'll want to install on it to make everything work. Things such as Cluster-Autoscaler, an ingress controller, a mesh network technology, various CSI volume provisioners, a runner for your chosen CI/CD platform, a disk volume autoscaler (shameless plug I wrote this) etc. Learn to deploy Helm charts on it, and learn to deploy some of your services onto it, exposing them to the internet. Learn to install and use Prometheus and Grafana on it to get in-depth metrics and visualization. Learn how to use Prometheus Alertmanager to trigger alerts to your email, webhooks, slack, etc. There's a lot to learn, and it may feel intimidating, but get the ball rolling and incrementally improve/expand your experience.
-
How do you guys on Mac M1's get around the annoying port forwarding issues with k8s + docker?
References: I use docker and Kubernetes daily. I currently manage numerous clusters and maintain pipelines for hundreds of microservices as I type this. I've been converting microservices into Docker images for companies hundreds if not thousands of times by now over the last bunches of years. I am also an avid and passionate open-source evangelist and Kubernetes/DevOps consultant. I author some Kubernetes controllers such as the Volume Autoscaler and have a set of Open Source Helm Charts and I love to contribute code/fixes wherever I run into issues.
-
Accessing the Underlying Node
Old justifications for this were to resize drives but all major cloud providers support handling the resizing operation for you now. You still need to trigger the resize. But with a controller like the Kubernetes Volume Autoscaler you don’t even need to do that!
-
Monthly 'Shameless Self Promotion' thread - 2023/01
An new open-source Kubernetes controller, the Kubernetes Volume Autoscaler, which auto-resizes your Persistent Volumes when they get almost full
-
Monthly 'Shameless Self Promotion' thread - 2022/11
Kubernetes Volume Autoscaler - An Kubernetes Controller to automatically scale up volumes (disks). I just recently released an update based on some feedback, adding Prometheus metrics support and fixing a few bugs
-
How do you prevent overprovisioning
Autoscale everything. There’s no over provisioning if it just provisions as needed. HPA and Cluster Autoscaler and for disks I wrote and use the Volume Autoscaler. Nodes disappear as needed and appear as needed. I generally even do spot instances in production. All assuming you are using a cloud provider.
-
What are some must-have, can’t-live-without 3rd party apps/tools you have installed in your k8s clusters?
Volume Autoscaler - Automatically scale up your disks size, keeping your costs low and allowing you to grow over time. Also making one less thing your sysops/devops person has to do. (Shameless plug, I wrote this)
autoscaler
-
Upgrading Hundreds of Kubernetes Clusters
We use Cluster Autoscaler to automatically adjust the number of nodes (cluster size) based on your actual usage to ensure efficiency. Additionally, we deploy Vertical and Horizontal Pod Autoscalers to scale your applications' resources as their needs change automatically.
-
Not Everything Is Google's Fault (Just Most Things)
> * Hetzner: cheap, good service, the finest pets in the world, no cattle
You can absolutely do cattle with Hetzner. They support imaging and immutable infrastructure. They don't have a native auto scaling equivalent, but if you're using Kubernetes, they have a cluster autoscaler: https://github.com/kubernetes/autoscaler/blob/master/cluster...
-
Kubernetes(K8s) Autoscaler — a detailed look at the design and implementation of VPA
Here we take the VPA as a starting point to analyze the design and implementation principles of the VPA in Autoscaler. The source code for this article is based on Autoscaler HEAD fbe25e1.
- Scaling with Karpenter and Empty Pod(A.k.a Overprovisioning)
-
Reducing Cloud Costs on Kubernetes Dev Envs
Autoscaling over EKS can be accomplished using either the cluster-autoscaler project or Karpenter. If you want to use Spot instances, consider using Karpenter, as it has better integrations with AWS for optimizing spot pricing and availability, minimizing interruptions, and falling back to on-demand nodes if no spot instances are available.
-
☸️ Managed Kubernetes : Our dev is on AWS, our prod is on OVH
Autoscaling is already provided on OVH, but we don't use it for now. Autoscaler has to be manually installed on the AWS/EKS cluster.
-
relevant way of scaling pods
do you mean this: https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/pkg/recommender/README.md
-
Kubernetes Cluster Maintenance
Read more about this scaler in detail here!
-
Anyone running Windows nodes in your clusters?
We have a default node group of Linux hosts, but there's a secondary nodegroup of Windows hosts that is typically scaled down to 0. When a team's build runs, a pod is scheduled based on their definition. Cluster-autoscaler will check the nodeSelector and automatically spin up a node from that nodegroup if necessary.
-
How to make sure Kubernetes autoscaler not deleting the nodes which runs specific pod
I am running a Kubernetes cluster(AWS EKS one) with Autoscaler pod So that Cluster will autoscale according to the resource request within the cluster.
What are some alternatives?
pvc-autoresizer - Auto-resize PersistentVolumeClaim objects based on Prometheus metrics
karpenter-provider-aws - Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Grafana-Dashboards - A variety of open-source Grafana dashboards typically for AWS and Kubernetes
cluster-proportional-autoscaler - Kubernetes Cluster Proportional Autoscaler Container
SparrowCI - SparrowCI - super fun and flexible CI system with many programming languages support
aws-ebs-csi-driver - CSI driver for Amazon EBS https://aws.amazon.com/ebs/
Helm-Chart-Boilerplates - Example implementations of the universal helm charts
keda - KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
sparrowci_web - ci.sparrowhub.io website
descheduler - Descheduler for Kubernetes
win2s3 - Windows to S3 Backup, Restore, Point in Time, and File Permissions
k3s-aws-terraform-cluster - Deploy an high available K3s cluster on Amazon AWS