Kubernetes with IPv6 on AWS

• cilium

  24 18,510 10.0 Go

  eBPF-based Networking, Security, and Observability

  

Taking the simpler private IP mode first, we found various issues with how various components decide which IP to use. E.g metrics-server will pick the first IP on the Node object regardless of what the Pod IP is. So ordering of the Node IPs matter. CNIs still show behavior that suggests it is not that well-tested yet. For example Cilium struggles with routing issues in this 18-months-old issue.

• systemd

  510 12,457 10.0 C

  The systemd System and Service Manager

  

For public IPs, there are some additional problems. On most Linux distro's accept_ra=2 sysctl must be set on the correct interfaces. And since the interface name depends on distro and instance type, this is a bit tricky. On Ubunutu, this is not need because Systemd has taken over a lot of the kernel responsibilities in this area. Systemd is not without bugs though, so when IPv6 single-address DHCPv6 is mixed with prefix delegation, DHCPv6 breaks. Hopefully this fix will make it into Ubuntu soon. Cilium works around this issue, but all other CNIs lose Node connectivity about 5 min after kOps configuration has finished.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• aws-load-balancer-controller

  39 3,754 8.9 Go

  A Kubernetes controller for Elastic Load Balancers

  

One of the benefits I mentioned above was using Pods as targets for load balancers. This is a feature that AWS Load Balancer Controller supports. But alas! AWS has two endpoints for the EC2 API. A single-stack IPv4 endpoint at ec2..amazonaws.com and a dual-stack one at https://api.ec2.eu-west-1.aws`. The SDK will use the former unless configured in code to use something else, and this is not currently possible. There is a pull request for this, but that only brings you to the next component. And if you want to use Cluster Autocaler you are also out of luck because AWS doesn't provide a dual-stack endpoint of the autoscaling API at all.

Suggest a related project