kotal VS cert-manager

https://kotal.co spin nodes on your infrastructure and create tls secure endpoints for you.

root@edgevpn1:~# kubectl apply -f https://github.com/kotalco/kotal/releases/download/v0.1-alpha.6/kotal.yaml namespace/kotal created customresourcedefinition.apiextensions.k8s.io/beaconnodes.ethereum2.kotal.io created customresourcedefinition.apiextensions.k8s.io/clusterpeers.ipfs.kotal.io created customresourcedefinition.apiextensions.k8s.io/nodes.chainlink.kotal.io created customresourcedefinition.apiextensions.k8s.io/nodes.ethereum.kotal.io created customresourcedefinition.apiextensions.k8s.io/nodes.filecoin.kotal.io created customresourcedefinition.apiextensions.k8s.io/nodes.polkadot.kotal.io created customresourcedefinition.apiextensions.k8s.io/peers.ipfs.kotal.io created customresourcedefinition.apiextensions.k8s.io/validators.ethereum2.kotal.io created mutatingwebhookconfiguration.admissionregistration.k8s.io/mutating-webhook-configuration created role.rbac.authorization.k8s.io/leader-election-role created clusterrole.rbac.authorization.k8s.io/manager-role created clusterrole.rbac.authorization.k8s.io/metrics-reader created clusterrole.rbac.authorization.k8s.io/proxy-role created rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/proxy-rolebinding created service/controller-manager-metrics-service created service/webhook-service created deployment.apps/controller-manager created certificate.cert-manager.io/serving-cert created issuer.cert-manager.io/selfsigned-issuer created validatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration created root@edgevpn1:~# kubectl get po,svc -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/local-path-provisioner-5b5579c644-pv9cj 1/1 Running 0 48m kube-system pod/coredns-75fc8f8fff-clcz6 1/1 Running 0 48m kube-system pod/helm-install-traefik-crd-tz9t9 0/1 Completed 0 48m kube-system pod/helm-install-traefik-bwc74 0/1 Completed 1 48m kube-system pod/svclb-traefik-eedbcbde-dj69n 2/2 Running 0 48m kube-system pod/traefik-9c6dc6686-nckxj 1/1 Running 0 48m kube-system pod/metrics-server-5c8978b444-ggqg7 1/1 Running 0 48m kube-system pod/svclb-traefik-eedbcbde-sb8h9 2/2 Running 0 33m cert-manager pod/cert-manager-77f9f95c5c-fk98d 1/1 Running 0 3m39s cert-manager pod/cert-manager-webhook-765b6f5ffc-fzj5p 1/1 Running 0 3m39s cert-manager pod/cert-manager-cainjector-769c877458-4wxmb 1/1 Running 2 (71s ago) 3m39s kotal pod/controller-manager-5784dc84d5-kbr58 2/2 Running 0 71s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 10.43.0.1 443/TCP 48m kube-system service/kube-dns ClusterIP 10.43.0.10 53/UDP,53/TCP,9153/TCP 48m kube-system service/metrics-server ClusterIP 10.43.26.165 443/TCP 48m kube-system service/traefik LoadBalancer 10.43.243.18 10.1.0.3,10.1.0.4 80:31735/TCP,443:30749/TCP 48m cert-manager service/cert-manager ClusterIP 10.43.12.233 9402/TCP 3m39s cert-manager service/cert-manager-webhook ClusterIP 10.43.217.216 443/TCP 3m39s kotal service/controller-manager-metrics-service ClusterIP 10.43.188.129 8443/TCP 71s kotal service/webhook-service ClusterIP 10.43.142.211 443/TCP 71s

we're developing a Kubernetes operator https://github.com/kotalco/kotal and we're testing it on 1.19 to 1.25 k8s clusters using KinD, by creating cluster, testing, then tearing it down.

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.25.8+k3s1 INSTALL_K3S_EXEC="--flannel-backend=none --disable-network-policy --disable=traefik --cluster-cidr=10.42.0.0/16" sh - Install kubectl from APT https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/ Install helm from APT https://helm.sh/docs/intro/install/ cp /etc/rancher/k3s/k3s.yaml .kube/config cp /etc/rancher/k3s/k3s.yaml /root/.kube/config kubectl create -f tigera-operator.yaml #Change ippools CIDR to 10.42.0.0/16 kubectl create -f custom-resources.yaml watch kubectl get pods --all-namespaces kubectl get nodes -o wide helm repo add rancher-stable https://releases.rancher.com/server-charts/stable kubectl create namespace cattle-system kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.1/cert-manager.crds.yaml helm repo add jetstack https://charts.jetstack.io helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.5.1 helm install rancher rancher-stable/rancher --namespace cattle-system --set hostname=scrapper.todoroff.net --set global.cattle.psp.enabled=false kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}' xxxxxxxxxxxxxxv6h72ckxp2xz2fpgqrlw864s2wjxbw8mwcr75

root@edgevpn1:~# kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.3/cert-manager.yaml customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created namespace/cert-manager created serviceaccount/cert-manager-cainjector created serviceaccount/cert-manager created serviceaccount/cert-manager-webhook created clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created clusterrole.rbac.authorization.k8s.io/cert-manager-view created clusterrole.rbac.authorization.k8s.io/cert-manager-edit created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created role.rbac.authorization.k8s.io/cert-manager:leaderelection created role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created service/cert-manager created service/cert-manager-webhook created deployment.apps/cert-manager-cainjector created deployment.apps/cert-manager created deployment.apps/cert-manager-webhook created mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created

I am using cert-manager 0.5.2 to manage Let's Encrypt certificates on our Kubernetes cluster.

helm repo add rancher-latest https://releases.rancher.com/server-charts/latest helm repo add jetstack https://charts.jetstack.io kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.6.1/cert-manager.crds.yaml helm upgrade -i cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace helm upgrade -i rancher rancher-latest/rancher --version 2.5.4 --create-namespace --namespace cattle-system --set hostname=rancher.zbs.local --set bootstrapPassword=bootStrapAllTheThings --set replicas=1

Waiting for http-01 challenge propagation: failed to perform self check GET request, it's similar to this bug https://github.com/jetstack/cert-manager/issues/656but all solutions from GitHub ticket comments didn't help.

- name: Finalize cluster become: no hosts: control[0] remote_user: ansibleadmin vars: tfconfig: "{{ lookup('file', 'variables.auto.tfvars.json') }}" tasks: - name: Remove taint to allow workloads on masters command: kubectl taint nodes --all=true node-role.kubernetes.io/master:NoSchedule- - name: Install helm ansible.builtin.shell: creates: /usr/local/bin/helm cmd: curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash - name: Install Rancher Helm repo command: helm repo add rancher-stable https://releases.rancher.com/server-charts/stable - name: Install jetstack Helm repo command: helm repo add jetstack https://charts.jetstack.io - name: Update helm repos command: helm repo update - name: Create rancher namespace command: kubectl create namespace cattle-system - name: Install cert manager CRD command: kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.crds.yaml - name: Install cert manager Helm chart command: helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.7.1 - name: Install Rancher via Helm command: helm install rancher rancher-stable/rancher --namespace cattle-system --set hostname={{ tfconfig['cluster-id'] }}-lb.{{ tfconfig['domain'] }} --set replicas=3 --set bootstrapPassword={{ tfconfig['rancher-password'] }} - name: Show boot pass debug: msg: "Visit this link in your browser to complete rancher setup: https://{{ tfconfig['cluster-id'] }}-lb.{{ tfconfig['domain'] }}/dashboard/?setup={{ tfconfig['rancher-password'] }}"