vault-k8s VS external-secrets

enabled: true # Use the Vault K8s Image https://github.com/hashicorp/vault-k8s/ image: repository: "hashicorp/vault-k8s" tag: "latest" resources: requests: memory: 256Mi cpu: 250m limits: memory: 256Mi cpu: 250m server: # These Resource Limits are in line with node requirements in the # Vault Reference Architecture for a Small Cluster resources: requests: memory: 8Gi cpu: 2000m limits: memory: 16Gi cpu: 2000m # For HA configuration and because we need to manually init the vault, # we need to define custom readiness/liveness Probe settings readinessProbe: enabled: true path: "/v1/sys/health?standbyok=true&sealedcode=204&uninitcode=204" livenessProbe: enabled: true path: "/v1/sys/health?standbyok=true" initialDelaySeconds: 60 # extraEnvironmentVars is a list of extra environment variables to set with the stateful set. These could be # used to include variables required for auto-unseal. extraEnvironmentVars: VAULT_CACERT: /vault/userconfig/tls-ca/ca.crt # extraVolumes is a list of extra volumes to mount. These will be exposed # to Vault in the path \/vault/userconfig//`. extraVolumes: - type: secret name: tls-ca - type: secret name: tls-listener-1 - type: secret name: tls-server # This configures the Vault Statefulset to create a PVC for audit logs. # See https://www.vaultproject.io/docs/audit/index.html to know more auditStorage: enabled: true storageClass: solidfire-gold dataStorage: enabled: true storageClass: solidfire-gold standalone: enabled: false # Run Vault in "HA" mode. ha: enabled: true replicas: 3 raft: enabled: true setNodeId: true config: |          ui = true          listener "tcp" {            address = "0.0.0.0:8200"            cluster_address = "0.0.0.0:8201"            tls_cert_file = "/vault/userconfig/tls-listener-1/server.crt"            tls_key_file = "/vault/userconfig/tls-listener-1/server.key"            tls_client_ca_file = "/vault/userconfig/tls-ca/ca.crt"          }          storage "raft" {            path = "/vault/data"            retry_join {              leader_api_addr = "https://vault-0.vault-internal:8200"`               leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"               leader_client_cert_file = "/vault/userconfig/tls-server/server.crt"               leader_client_key_file = "/vault/userconfig/tls-server/server.key"             }             retry_join {               leader_api_addr = "https://vault-1.vault-internal:8200"               leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"               leader_client_cert_file = "/vault/userconfig/tls-server/server.crt"               leader_client_key_file = "/vault/userconfig/tls-server/server.key"             }             retry_join {               leader_api_addr = "https://vault-2.vault-internal:8200"               leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"               leader_client_cert_file = "/vault/userconfig/tls-server/server.crt"               leader_client_key_file = "/vault/userconfig/tls-server/server.key"             }             autopilot {               cleanup_dead_servers = "true"               last_contact_threshold = "200ms"               last_contact_failure_threshold = "10m"               max_trailing_logs = 250000               min_quorum = 3               server_stabilization_time = "10s"             }           }           service_registration "kubernetes" {} # Vault UI ui: enabled: true serviceType: "LoadBalancer" externalPort: 8443 loadBalancerIP: 10.193.124.70 loadBalancerSourceRanges: - 0.0.0.0/0

vault: global: enabled: true tlsDisable: false injector: enabled: true # Use the Vault K8s Image https://github.com/hashicorp/vault-k8s/ image: repository: "hashicorp/vault-k8s" tag: "latest" resources: requests: memory: 256Mi cpu: 250m limits: memory: 256Mi cpu: 250m server: # These Resource Limits are in line with node requirements in the # Vault Reference Architecture for a Small Cluster resources: requests: memory: 8Gi cpu: 2000m limits: memory: 16Gi cpu: 2000m # For HA configuration and because we need to manually init the vault, # we need to define custom readiness/liveness Probe settings readinessProbe: enabled: true path: "/v1/sys/health?standbyok=true&sealedcode=204&uninitcode=204" livenessProbe: enabled: true path: "/v1/sys/health?standbyok=true" initialDelaySeconds: 60 # extraEnvironmentVars is a list of extra environment variables to set with the stateful set. These could be # used to include variables required for auto-unseal. extraEnvironmentVars: VAULT_CACERT: /vault/userconfig/tls-ca/ca.crt # extraVolumes is a list of extra volumes to mount. These will be exposed # to Vault in the path \/vault/userconfig//`. extraVolumes: - type: secret name: tls-listener - type: secret name: tls-server - type: secret name: tls-ca # This configures the Vault Statefulset to create a PVC for audit logs. # See https://www.vaultproject.io/docs/audit/index.html to know more auditStorage: enabled: true storageClass: solidfire-gold dataStorage: enabled: true storageClass: solidfire-gold standalone: enabled: false # Run Vault in "HA" mode. ha: enabled: true replicas: 3 raft: enabled: true setNodeId: true config: |          ui = true          listener "tcp" {            address = "[::]:8200"            cluster_address = "[::]:8201"            tls_cert_file = "/vault/userconfig/tls-listener/server.crt"            tls_key_file = "/vault/userconfig/tls-listener/server.key"            tls_client_ca_file = "/vault/userconfig/tls-ca/ca.crt"          }          storage "raft" {            path = "/vault/data"            retry_join {              leader_api_addr = "https://vault-0.vault-internal:8200"`               leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"               leader_client_cert_file = "/vault/userconfig/tls-server/server.crt"               leader_client_key_file = "/vault/userconfig/tls-server/server.key"             }             retry_join {               leader_api_addr = "https://vault-1.vault-internal:8200"               leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"               leader_client_cert_file = "/vault/userconfig/tls-server/server.crt"               leader_client_key_file = "/vault/userconfig/tls-server/server.key"             }             retry_join {               leader_api_addr = "https://vault-2.vault-internal:8200"               leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"               leader_client_cert_file = "/vault/userconfig/tls-server/server.crt"               leader_client_key_file = "/vault/userconfig/tls-server/server.key"             }             autopilot {               cleanup_dead_servers = "true"               last_contact_threshold = "200ms"               last_contact_failure_threshold = "10m"               max_trailing_logs = 250000               min_quorum = 5               server_stabilization_time = "10s"             }           }           service_registration "kubernetes" {} # Vault UI ui: enabled: true serviceType: "LoadBalancer" serviceNodePort: null externalPort: 8200

Hashicorp Vault k8s is an operator that modifies pods via a mutating webhook to connect between vault and pod via sidecars (additional containers) to provide secrets. This has the major advantage that no secret objects are created in Kubernetes here. The disadvantage is that this way only works with Vault.

They’ve made it so you can define the order that the vault sidecar starts in, so that the proxy will be running first. https://github.com/hashicorp/vault-k8s/issues/53

Something I personally like about EKS is the Amazon Controllers for Kubernetes nowadays they would more preoperly be called 'operators' like the (non AWS and non AWS specific) External Secrets Operator. Essentially you delegate your cluster to create external resources elsewhere on your behalf based on annotations in your deployment.

Vault is extremely complex and heavy for my tastes, and Bitwarden Secrets Manager implementation AFAIU is not open source and not suitable for self-hosting. I like that both can be easily integrated with External Secrets for kubernetes secrets management.

Store the Secrets in a vault like Hashicorp Vault, AWS Secrets Manager, GCP Secret Manager, etc., and then use an operator like External Secrets Operator to add them to your K8s cluster.

External Secrets is an operator that integrates external KMS such as Hashicorp Vault or those of the major cloud providers. It reads secrets from the external APIs and injects them into Kubernetes secrets. The operator is a new implementation after the merge of similar projects from GoDaddy and ContainerSolutions.

I use Kubernetes-External-Secrets https://github.com/external-secrets/external-secrets with aws parameter store