aws-efs-csi-driver VS amazon-eks-pod-identity-webhook

In the past I was have problems with GID allocator, something related to this problem.

However, the docs (https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/examples/kubernetes/dynamic_provisioning/README.md) are telling me to create EFS Mount Targets in the EKS subnets. Thats fine.

Fargate storage A Pod running on Fargate automatically mounts an Amazon EFS file system. You can't use dynamic persistent volume provisioning with Fargate nodes, but you can use static provisioning. For more information, see Amazon EFS CSI Driver on GitHub.

I guess something like this might go a long way to solve the problem https://github.com/kubernetes-sigs/aws-efs-csi-driver/pull/640 ? Though I see it isn't merged yet

I am currently trying to create an EFS for use within an EKS cluster. I've followed all the instructions, and everything seems to be working for the most part. However, when trying to apply the multiple_pods example deployment from here, the pods cannot succesfully mount the file system. The PV and PVC are both bound and look good, however the pods do not start and yield the following error message:

In each namespace, create a PV/PVC using the same fixed volume path. See "Volume Path in EKS CSI Driver" To make this work however, you MUST pre-create this volume path in your EFS (I usually just have an EC2 instance with it mounted to work on). From the docs above "Note: this feature requires the sub directory to mount precreated on EFS before consuming the volume from"

resource "aws_iam_policy" "eks_efs_csi_driver_policy" { # https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/iam-policy-example.json policy = file("./6.AWSEFSpolicy.json") name = "aws-efs-csi-policy" }

efs - IAM Policy for AWS EFS CSI Driver.

Its not specific to EKS, you can find the underlying webhook that injects the "identity" here: https://github.com/aws/amazon-eks-pod-identity-webhook

You have to jump through much of the same hoops you describe, having a public `.well-known` endpoint for example. I have achieved this in the past by putting the OIDC discovery information in an S3 bucket.

You can install the pod identity webhook and AWS cloud provider, csi provider etc on a bare kube cluster and get pretty close to the EKS experience. Not something I’d do for prod, but interesting as a learning exercise.

Yes you absolutely can. https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md

Is your pod running as an unprivileged user? Sounds like https://github.com/aws/amazon-eks-pod-identity-webhook/issues/8 to me.

On EKS, the pod identity webhook is commonly used as the mechanism for adding the necessary parts of the Pod spec. This webhook looks for ServiceAccounts with a specific set of annotations telling it what ARN it can assume and various other settings. When a Pod is created that uses one of these ServiceAccounts, the webhook mutates the Pod using information found in the ServiceAccount annotations.

If you prefer, you could create ServiceAccounts with these details and use the EKS identity webhook, but I don't see kOps supporting that webhook as a native addon.

The Amazon EKS Pod Identity Webhook on the cluster watches for pods that are associated with service accounts with this special annotation & injects Web Identity Token credentials into the pod as environment variables (technical details here).

I don’t know how KOPs manages IAM creds for pods these days, but you can use this (my recommendation) https://github.com/aws/amazon-eks-pod-identity-webhook, or something like KIAM or kube2iam