eksctl VS prometheus

There are a variety of ways in which you can create an Amazon EKS cluster. I prefer using eksctl CLI because of the convenience it offers. Creating an an EKS cluster using eksctl, can be as easy as this:

There are many ways to create a cluster such as using eksctl. In my case, I will use terraform module cause it’s easy to reuse and comprehend.

eksctl [eksctl] is the tool that can provision EKS cluster as well as supporting VPC network infrastructure.

For this and many other reasons I recommend doing everything in Terraform EXCEPT EKS and its node groups. For that, I use https://eksctl.io/ because it much better manages the lifecycle of EKS and your node groups. I have an blog article better explaining why I recommend it, and another blog article explaining how to do zero-downtime upgrades with EKSCTL.

eksctl

Second, make sure you create a spot instance group that attempts to launch MULTIPLE different instance types. This way if one instance type gets flushed, your autoscaler will kick in and launch a different type. Without this, you WILL HAVE DOWNTIME if a sudden price hike and flush occurs. If you're using eksctl I have example configurations that use multi-instance types on Github here.

There are a variety of ways in which you can create an Amazon EKS cluster. I prefer using eksctl CLI because of the convenience it offers. Creating an an EKS cluster using eksctl, can be as easy as this:

I've written an article on this, with my recommended tool for managing eks EKSCTL.

--- AWSTemplateFormatVersion: '2010-09-09' Parameters: VpcId: Type: AWS::EC2::VPC::Id Description: ID of the VPC in which to create the Kubernetes cluster SubnetIds: Type: List Description: List of Subnet IDs in which to create the Kubernetes cluster KeyPairName: Type: AWS::EC2::KeyPair::KeyName Description: Name of the EC2 Key Pair to use for SSH access to worker nodes ClusterName: Type: String Description: Name of the Kubernetes cluster to create Resources: ControlPlaneSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref VpcId GroupDescription: Allow inbound traffic to the Kubernetes control plane SecurityGroupIngress: - IpProtocol: tcp FromPort: 22 ToPort: 22 CidrIp: 0.0.0.0/0 WorkerNodeSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref VpcId GroupDescription: Allow inbound traffic to Kubernetes worker nodes SecurityGroupIngress: - IpProtocol: tcp FromPort: 22 ToPort: 22 CidrIp: 0.0.0.0/0 ControlPlaneInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Roles: - !Ref ControlPlaneRole ControlPlaneRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/AmazonEKSClusterPolicy - arn:aws:iam::aws:policy/AmazonEKSServicePolicy ControlPlaneInstance: Type: AWS::EC2::Instance Properties: ImageId: ami-0b69ea66ff7391e80 InstanceType: t2.micro KeyName: !Ref KeyPairName NetworkInterfaces: - DeviceIndex: 0 AssociatePublicIpAddress: true GroupSet: - !Ref ControlPlaneSecurityGroup SubnetId: !Select [0, !Ref SubnetIds] IamInstanceProfile: !Ref ControlPlaneInstanceProfile UserData: Fn::Base64: !Sub | #!/bin/bash echo 'net.bridge.bridge-nf-call-iptables=1' | tee -a /etc/sysctl.conf sysctl -p yum update -y amazon-linux-extras install docker -y service docker start usermod -a -G docker ec2-user curl -o /usr/local/bin/kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/linux/amd64/kubectl chmod +x /usr/local/bin/kubectl echo 'export PATH=$PATH:/usr/local/bin' >> /etc/bashrc curl --silent --location "https://github.com/weaveworks/eksctl/releases

It's like Prometheus, but for logs. Okay it's not really to do with the Norse or Greek gods, instead Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by the open source project Prometheus. Built by Grafana Labs, Loki is designed for ease of use. Instead of indexing the contents of the logs, Loki provides a set of labels for each log stream. The latest update includes query acceleration with Bloom filters, native OTel support, Helm charts, and more. Check out the changelog for all the major changes and deprecations.

WriteRequest::timeseries is a vector (https://github.com/prometheus/prometheus/blob/main/prompb/re...) and

Developers widely use Prometheus as a system for operational monitoring and alerting for their projects. Here is a list of tools for monitoring frontend services with Prometheus.

Just to give an example of the power of Go for CLI builds, you may have already used or at least heard of Docker, Kubernetes, Prometheus, Terraform, but what do they all have in common? They all have a large part of their usability via CLI and are developed in Go 🐿.

Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

Setting up monitoring for a system, especially one involving GRPC communication, provides crucial visibility into its operations. In this guide, we walked through the steps to instrument both a GRPC server and client with Prometheus metrics, exposed those metrics via an HTTP endpoint, and visualized them using Grafana. The Docker-Compose setup simplified the deployment of both Prometheus and Grafana, ensuring a streamlined process.

Alerting and Notification: Select a tool with flexible alerting mechanisms to proactively detect anomalies or deviations from defined thresholds. Consider asking questions like "Does this tool offer customizable alerting options and support notification channels that suit our team's communication preferences?" A tool like Prometheus provides robust alerting capabilities.

Prometheus

Prometheus is an open-source monitoring and alerting toolkit. It is designed to monitor highly dynamic containerized systems, making it an excellent choice for monitoring Docker containers and Kubernetes clusters.

wget https://github.com/prometheus/prometheus/releases/download/v2.46.0/prometheus-2.46.0.linux-amd64.tar.gz