terraform-aws-eks VS external-dns

cloudposse module was popular but most have moved to https://github.com/terraform-aws-modules/terraform-aws-eks also eks blueprints will be moving to this module. use eks blueprints v5

That's a very simplistic view. Let's do a small thought exercise. Is this module not infrastructure?

I think there is an issue with the module eks : https://github.com/terraform-aws-modules/terraform-aws-eks

module "eks" { source = "terraform-aws-modules/eks/aws" cluster_name = var.cluster_name cluster_version = var.kubernetes_version cluster_endpoint_private_access = true cluster_endpoint_public_access = true cluster_addons = { coredns = { most_recent = true timeouts = { create = "2m" # default 20m. Times out on first launch while being effectively created } } kube-proxy = { most_recent = true } vpc-cni = { most_recent = true } aws-ebs-csi-driver = { most_recent = true } } vpc_id = module.vpc.vpc_id subnet_ids = module.vpc.private_subnets # Self managed node groups will not automatically create the aws-auth configmap so we need to create_aws_auth_configmap = true manage_aws_auth_configmap = true aws_auth_users = var.aws_auth_users enable_irsa = true node_security_group_additional_rules = { ingress_self_all = { description = "Node to node all ports/protocols" protocol = "-1" from_port = 0 to_port = 0 type = "ingress" self = true } egress_all = { # by default, only https urls can be reached from inside the cluster description = "Node all egress" protocol = "-1" from_port = 0 to_port = 0 type = "egress" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } } self_managed_node_group_defaults = { # enable discovery of autoscaling groups by cluster-autoscaler autoscaling_group_tags = { "k8s.io/cluster-autoscaler/enabled" : true, "k8s.io/cluster-autoscaler/${var.cluster_name}" : "owned", } # from https://github.com/terraform-aws-modules/terraform-aws-eks/issues/2207#issuecomment-1220679414 # to avoid "waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator" iam_role_additional_policies = { AmazonEBSCSIDriverPolicy = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy" } } # possible values : https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/node_groups.tf self_managed_node_groups = { default_node_group = { create = false } # fulltime-az-a = { # name = "fulltime-az-a" # subnets = [module.vpc.private_subnets[0]] # instance_type = "t3.medium" # desired_size = 1 # bootstrap_extra_args = "--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=normal'" # } spot-az-a = { name = "spot-az-a" subnet_ids = [module.vpc.private_subnets[0]] # only one subnet to simplify PV usage # availability_zones = ["${var.region}a"] # conflict with previous option. TODO try subnet_ids=null at creation (because at modification it fails) desired_size = 2 min_size = 1 max_size = 10 bootstrap_extra_args = "--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'" use_mixed_instances_policy = true mixed_instances_policy = { instances_distribution = { on_demand_base_capacity = 0 on_demand_percentage_above_base_capacity = 0 spot_allocation_strategy = "lowest-price" # "capacity-optimized" described here : https://aws.amazon.com/blogs/compute/introducing-the-capacity-optimized-allocation-strategy-for-amazon-ec2-spot-instances/ } override = [ { instance_type = "t3.xlarge" weighted_capacity = "1" }, { instance_type = "t3a.xlarge" weighted_capacity = "1" }, ] } } } tags = local.tags }

If you want somewhat viable setup - I'd go for terraform-aws-modules (Anton did an awesome job), and aws-ia blueprints, especially those multi-tenant ones.

‏https://github.com/terraform-aws-modules/terraform-aws-eks

I tried using explicit depends_on between my modules but this practise is not recommended since it cause issues during planning.

If you use https://github.com/terraform-aws-modules/terraform-aws-eks it is designed to upgrade the components in the correct order when the cluster version is changed

The second one is a combination of tools: External DNS, cert-manager, and NGINX ingress. Using these as a stack, you can quickly deploy an application, making it available through a DNS with a TLS without much effort via simple annotations. When I first discovered External DNS, I was amazed at its quality.

One of the reasons why I chose Hetzner was that it WAS supported by the ExternalDNS project. I didn't quite understand why the Hetzner provider was pulled, but I saw that an attempt of re-adding it was refused, on the ground that the upcoming webhook architecture would have allowed to better maintain providers.

Write a custom controller for the external DNS controller, or setup some form of ArgoCD app / appset templating.

so I am looking at having an automated way for new routers registered in Traefik to also have the corresponding DNS entry added to my Pihole instance similar to external-dns but obviously, this is exclusive to ingress on k8s environments. my current setup is traefik in a container on unraid.

You technically don’t need to run a load balancer or have a virtual IP for your control plane. If you control your dns, you can add an A record pointing to all IPs for your control plane nodes. It won’t load balance your traffic, but combined with something like External DNS it gives you HA for the control plane.

I normally deploy external-dns, which automatically updates DNS with the ingress controller's external IP address.

Background: Having a look I can see this https://github.com/kubernetes-sigs/external-dns

3) Use the Kubernetes External-DNS. I've never used this, but this is assuming it can update DNS for each pods/app to point to the correct Node (it'd need to update my Homelab DNS running on Windows Server)

So here’s the solution I have taken for our current stack. EKS and its dependencies are created through terraform using the eks module as well as provision a route53 subdomain and a wildcard cert. Once we have that created, I have installed this deployment into the cluster via the helm module: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/. This allows me to use kuberentes resources (load balancers or ingress objects) and it will handle all the provisioning of load balancers and security groups for me, based on my application yaml and annotations. We also use https://github.com/kubernetes-sigs/external-dns to manage all of our specific host names for the applications through annotations. So to generally put, terraform manages out Kubernetes clusters, and Kubernetes manages the deployment of anything needed for the application including volumes, load balancers, hostnames though Kubernetes system deployments

Metallb for your load balancer (layer2 mode will do) NginX-ingress, will be spot on for internal home apps External-dns to publish your dns record to your Dns server at home, https://github.com/kubernetes-sigs/external-dns