virtink VS k3s-ansible

$ kubectl apply -f https://github.com/smartxworks/virtink/releases/download/v0.11.0/virtink.yaml  ✔   namespace/virtink-system created customresourcedefinition.apiextensions.k8s.io/virtualmachinemigrations.virt.virtink.smartx.com created customresourcedefinition.apiextensions.k8s.io/virtualmachines.virt.virtink.smartx.com created serviceaccount/virt-controller created serviceaccount/virt-daemon created clusterrole.rbac.authorization.k8s.io/virt-controller created clusterrole.rbac.authorization.k8s.io/virt-daemon created clusterrolebinding.rbac.authorization.k8s.io/virt-controller created clusterrolebinding.rbac.authorization.k8s.io/virt-daemon created service/virt-controller created deployment.apps/virt-controller created daemonset.apps/virt-daemon created certificate.cert-manager.io/virt-controller-cert created certificate.cert-manager.io/virt-daemon-cert created issuer.cert-manager.io/virt-controller-cert-issuer created issuer.cert-manager.io/virt-daemon-cert-issuer created mutatingwebhookconfiguration.admissionregistration.k8s.io/virtink-mutating-webhook-configuration created validatingwebhookconfiguration.admissionregistration.k8s.io/virtink-validating-webhook-configuration created $ kubectl get po,svc -A  ✔  NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/local-path-provisioner-7b7dc8d6f5-8cwd8 1/1 Running 0 10m kube-system pod/coredns-b96499967-cc7fw 1/1 Running 0 10m kube-system pod/svclb-traefik-baea9d36-vjxzg 2/2 Running 0 10m kube-system pod/helm-install-traefik-crd-ghbn2 0/1 Completed 0 10m kube-system pod/svclb-traefik-baea9d36-2vw6x 2/2 Running 0 10m kube-system pod/helm-install-traefik-fc9xw 0/1 Completed 1 10m kube-system pod/svclb-traefik-baea9d36-g42x2 2/2 Running 0 10m kube-system pod/traefik-56cfcbb99f-5qgvd 1/1 Running 0 10m kube-system pod/metrics-server-84f8d4c4fc-wx8bl 1/1 Running 0 10m cert-manager pod/cert-manager-cainjector-5987875fc7-g94ft 1/1 Running 0 2m58s cert-manager pod/cert-manager-6dd9658548-slxrp 1/1 Running 0 2m58s cert-manager pod/cert-manager-webhook-7b4c5f579b-76x57 1/1 Running 0 2m58s virtink-system pod/virt-daemon-dv9m9 1/1 Running 0 41s virtink-system pod/virt-daemon-mv994 1/1 Running 0 41s virtink-system pod/virt-controller-768b979d4-gzlzt 1/1 Running 0 41s virtink-system pod/virt-daemon-jpz87 1/1 Running 0 41s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 10.43.0.1 443/TCP 11m kube-system service/kube-dns ClusterIP 10.43.0.10 53/UDP,53/TCP,9153/TCP 11m kube-system service/metrics-server ClusterIP 10.43.21.66 443/TCP 10m kube-system service/traefik LoadBalancer 10.43.170.138 174.138.1.202,188.166.109.175,188.166.88.194 80:32630/TCP,443:30841/TCP 10m cert-manager service/cert-manager ClusterIP 10.43.182.103 9402/TCP 2m59s cert-manager service/cert-manager-webhook ClusterIP 10.43.136.111 443/TCP 2m58s virtink-system service/virt-controller ClusterIP 10.43.196.10 443/TCP 42s

Virtink is a Kubernetes add-on for running Cloud Hypervisor virtual machines. By using Cloud Hypervisor as the underlying hypervisor, Virtink enables a lightweight and secure way to run fully virtualized workloads in a canonical Kubernetes cluster.

git clone https://github.com/k3s-io/k3s-ansible.git cd k3s-ansible cp -R inventory/sample inventory/my-cluster nano inventory/my-cluster/hosts.ini

If you're down for some purchases, I recommend wholeheartedly to get the turing pi https://turingpi.com/ some raspberry modules and k3s-ansible https://github.com/k3s-io/k3s-ansible

I tried k3s recently and installing it was easy using k3s-ansible https://github.com/k3s-io/k3s-ansible

I use some ansible playbooks and when I am ready to update the version I update the variable in the playbooks. https://github.com/k3s-io/k3s-ansible

root@k3s1:~# git clone https://github.com/k3s-io/k3s-ansible Cloning into 'k3s-ansible'... remote: Enumerating objects: 922, done. remote: Total 922 (delta 0), reused 0 (delta 0), pack-reused 922 Receiving objects: 100% (922/922), 116.25 KiB | 3.32 MiB/s, done. Resolving deltas: 100% (351/351), done. root@k3s1:~# cd k3s-ansible/ root@k3s1:~/k3s-ansible# cp -R inventory/sample inventory/my-cluster root@k3s1:~/k3s-ansible# cat inventory/my-cluster/group_vars/all.yml --- k3s_version: v1.24.7+k3s1 ansible_user: ubuntu systemd_dir: /etc/systemd/system master_ip: "{{ hostvars[groups['master'][0]]['ansible_host'] | default(groups['master'][0]) }}" extra_server_args: "" extra_agent_args: "" root@k3s1:~/k3s-ansible# cat inventory/my-cluster/hosts.ini [master] 10.42.2.6 [node] 10.42.0.7 10.42.1.9 [k3s_cluster:children] master node root@k3s1:~/k3s-ansible# ssh-copy-id [email protected] /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys [email protected]'s password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh '[email protected]'" and check to make sure that only the key(s) you wanted were added. root@k3s1:~/k3s-ansible# ssh-copy-id [email protected] /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys [email protected]'s password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh '[email protected]'" and check to make sure that only the key(s) you wanted were added. root@k3s1:~/k3s-ansible# ssh-copy-id [email protected] /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys [email protected]'s password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh '[email protected]'" and check to make sure that only the key(s) you wanted were added. root@k3s1:~/k3s-ansible# apt install python3-pip -y && pip install ansible root@k3s1:~/k3s-ansible# ansible-playbook site.yml -i inventory/my-cluster/hosts.ini PLAY [k3s_cluster] **************************************************************************************************************************************************************** TASK [Gathering Facts] ************************************************************************************************************************************************************ Thursday 10 November 2022 23:20:03 +0000 (0:00:00.013) 0:00:00.013 ***** ok: [10.42.2.6] ok: [10.42.1.9] ok: [10.42.0.7] TASK [prereq : Set SELinux to disabled state] ************************************************************************************************************************************* Thursday 10 November 2022 23:20:04 +0000 (0:00:01.535) 0:00:01.548 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [prereq : Enable IPv4 forwarding] ******************************************************************************************************************************************** Thursday 10 November 2022 23:20:04 +0000 (0:00:00.058) 0:00:01.607 ***** changed: [10.42.2.6] changed: [10.42.1.9] changed: [10.42.0.7] TASK [prereq : Enable IPv6 forwarding] ******************************************************************************************************************************************** Thursday 10 November 2022 23:20:05 +0000 (0:00:00.318) 0:00:01.925 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [prereq : Add br_netfilter to /etc/modules-load.d/] ************************************************************************************************************************** Thursday 10 November 2022 23:20:05 +0000 (0:00:00.053) 0:00:01.978 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [prereq : Load br_netfilter] ************************************************************************************************************************************************* Thursday 10 November 2022 23:20:05 +0000 (0:00:00.051) 0:00:02.030 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [prereq : Set bridge-nf-call-iptables (just to be sure)] ********************************************************************************************************************* Thursday 10 November 2022 23:20:05 +0000 (0:00:00.052) 0:00:02.082 ***** skipping: [10.42.2.6] => (item=net.bridge.bridge-nf-call-iptables) skipping: [10.42.2.6] => (item=net.bridge.bridge-nf-call-ip6tables) skipping: [10.42.0.7] => (item=net.bridge.bridge-nf-call-iptables) skipping: [10.42.0.7] => (item=net.bridge.bridge-nf-call-ip6tables) skipping: [10.42.1.9] => (item=net.bridge.bridge-nf-call-iptables) skipping: [10.42.1.9] => (item=net.bridge.bridge-nf-call-ip6tables) TASK [prereq : Add /usr/local/bin to sudo secure_path] **************************************************************************************************************************** Thursday 10 November 2022 23:20:05 +0000 (0:00:00.072) 0:00:02.154 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [download : Download k3s binary x64] ***************************************************************************************************************************************** Thursday 10 November 2022 23:20:05 +0000 (0:00:00.103) 0:00:02.258 ***** [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually changed: [10.42.2.6] changed: [10.42.0.7] changed: [10.42.1.9] TASK [download : Download k3s binary arm64] *************************************************************************************************************************************** Thursday 10 November 2022 23:20:15 +0000 (0:00:10.036) 0:00:12.295 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [download : Download k3s binary armhf] *************************************************************************************************************************************** Thursday 10 November 2022 23:20:15 +0000 (0:00:00.067) 0:00:12.362 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [raspberrypi : Test for raspberry pi /proc/cpuinfo] ************************************************************************************************************************** Thursday 10 November 2022 23:20:15 +0000 (0:00:00.084) 0:00:12.447 ***** ok: [10.42.2.6] ok: [10.42.0.7] ok: [10.42.1.9] TASK [raspberrypi : Test for raspberry pi /proc/device-tree/model] **************************************************************************************************************** Thursday 10 November 2022 23:20:16 +0000 (0:00:00.359) 0:00:12.806 ***** ok: [10.42.2.6] ok: [10.42.0.7] ok: [10.42.1.9] TASK [raspberrypi : Set raspberry_pi fact to true] ******************************************************************************************************************************** Thursday 10 November 2022 23:20:16 +0000 (0:00:00.255) 0:00:13.062 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [raspberrypi : Set detected_distribution to Raspbian] ************************************************************************************************************************ Thursday 10 November 2022 23:20:16 +0000 (0:00:00.060) 0:00:13.123 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [raspberrypi : Set detected_distribution to Raspbian (ARM64 on Debian Buster)] *********************************************************************************************** Thursday 10 November 2022 23:20:16 +0000 (0:00:00.112) 0:00:13.235 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [raspberrypi : Set detected_distribution_major_version] ********************************************************************************************************************** Thursday 10 November 2022 23:20:16 +0000 (0:00:00.073) 0:00:13.309 ***** skipping: [10.42.2.6] skipping: [10.42.0.7] skipping: [10.42.1.9] TASK [raspberrypi : execute OS related tasks on the Raspberry Pi] ***************************************************************************************************************** Thursday 10 November 2022 23:20:16 +0000 (0:00:00.080) 0:00:13.389 ***** skipping: [10.42.2.6] => (item=/root/k3s-ansible/roles/raspberrypi/tasks/prereq/Ubuntu.yml) skipping: [10.42.0.7] => (item=/root/k3s-ansible/roles/raspberrypi/tasks/prereq/Ubuntu.yml) skipping: [10.42.1.9] => (item=/root/k3s-ansible/roles/raspberrypi/tasks/prereq/Ubuntu.yml) PLAY [master] ********************************************************************************************************************************************************************* TASK [Gathering Facts] ************************************************************************************************************************************************************ Thursday 10 November 2022 23:20:16 +0000 (0:00:00.125) 0:00:13.515 ***** ok: [10.42.2.6] TASK [k3s/master : Copy K3s service file] ***************************************************************************************************************************************** Thursday 10 November 2022 23:20:17 +0000 (0:00:00.660) 0:00:14.175 ***** changed: [10.42.2.6] TASK [k3s/master : Enable and check K3s service] ********************************************************************************************************************************** Thursday 10 November 2022 23:20:18 +0000 (0:00:00.688) 0:00:14.864 ***** changed: [10.42.2.6] TASK [k3s/master : Wait for node-token] ******************************************************************************************************************************************* Thursday 10 November 2022 23:20:29 +0000 (0:00:11.160) 0:00:26.025 ***** ok: [10.42.2.6] TASK [k3s/master : Register node-token file access mode] ************************************************************************************************************************** Thursday 10 November 2022 23:20:29 +0000 (0:00:00.458) 0:00:26.483 ***** ok: [10.42.2.6] TASK [k3s/master : Change file access node-token] ********************************************************************************************************************************* Thursday 10 November 2022 23:20:30 +0000 (0:00:00.329) 0:00:26.812 ***** changed: [10.42.2.6] TASK [k3s/master : Read node-token from master] *********************************************************************************************************************************** Thursday 10 November 2022 23:20:30 +0000 (0:00:00.422) 0:00:27.234 ***** ok: [10.42.2.6] TASK [k3s/master : Store Master node-token] *************************************************************************************************************************************** Thursday 10 November 2022 23:20:30 +0000 (0:00:00.303) 0:00:27.537 ***** ok: [10.42.2.6] TASK [k3s/master : Restore node-token file access] ******************************************************************************************************************************** Thursday 10 November 2022 23:20:30 +0000 (0:00:00.041) 0:00:27.579 ***** changed: [10.42.2.6] TASK [k3s/master : Create directory .kube] **************************************************************************************************************************************** Thursday 10 November 2022 23:20:31 +0000 (0:00:00.232) 0:00:27.812 ***** changed: [10.42.2.6] TASK [k3s/master : Copy config file to user home directory] *********************************************************************************************************************** Thursday 10 November 2022 23:20:31 +0000 (0:00:00.310) 0:00:28.122 ***** changed: [10.42.2.6] TASK [k3s/master : Replace https://localhost:6443 by https://master-ip:6443] ****************************************************************************************************** Thursday 10 November 2022 23:20:31 +0000 (0:00:00.211) 0:00:28.334 ***** changed: [10.42.2.6] TASK [k3s/master : Create kubectl symlink] **************************************************************************************************************************************** Thursday 10 November 2022 23:20:32 +0000 (0:00:00.693) 0:00:29.027 ***** changed: [10.42.2.6] TASK [k3s/master : Create crictl symlink] ***************************************************************************************************************************************** Thursday 10 November 2022 23:20:32 +0000 (0:00:00.187) 0:00:29.215 ***** changed: [10.42.2.6] PLAY [node] *********************************************************************************************************************************************************************** TASK [Gathering Facts] ************************************************************************************************************************************************************ Thursday 10 November 2022 23:20:32 +0000 (0:00:00.209) 0:00:29.424 ***** ok: [10.42.1.9] ok: [10.42.0.7] TASK [k3s/node : Copy K3s service file] ******************************************************************************************************************************************* Thursday 10 November 2022 23:20:33 +0000 (0:00:00.724) 0:00:30.149 ***** changed: [10.42.1.9] changed: [10.42.0.7] TASK [k3s/node : Enable and check K3s service] ************************************************************************************************************************************ Thursday 10 November 2022 23:20:34 +0000 (0:00:00.538) 0:00:30.687 ***** changed: [10.42.1.9] changed: [10.42.0.7] PLAY RECAP ************************************************************************************************************************************************************************ 10.42.0.7 : ok=8 changed=4 unreachable=0 failed=0 skipped=13 rescued=0 ignored=0 10.42.1.9 : ok=8 changed=4 unreachable=0 failed=0 skipped=13 rescued=0 ignored=0 10.42.2.6 : ok=19 changed=11 unreachable=0 failed=0 skipped=13 rescued=0 ignored=0 Thursday 10 November 2022 23:20:44 +0000 (0:00:10.395) 0:00:41.082 ***** =============================================================================== k3s/master : Enable and check K3s service --------------------------------------------------------------------------------------------------------------------------------- 11.16s k3s/node : Enable and check K3s service ----------------------------------------------------------------------------------------------------------------------------------- 10.40s download : Download k3s binary x64 ---------------------------------------------------------------------------------------------------------------------------------------- 10.04s Gathering Facts ------------------------------------------------------------------------------------------------------------------------------------------------------------ 1.54s Gathering Facts ------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.72s k3s/master : Replace https://localhost:6443 by https://master-ip:6443 ------------------------------------------------------------------------------------------------------ 0.69s k3s/master : Copy K3s service file ----------------------------------------------------------------------------------------------------------------------------------------- 0.69s Gathering Facts ------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.66s k3s/node : Copy K3s service file ------------------------------------------------------------------------------------------------------------------------------------------- 0.54s k3s/master : Wait for node-token ------------------------------------------------------------------------------------------------------------------------------------------- 0.46s k3s/master : Change file access node-token --------------------------------------------------------------------------------------------------------------------------------- 0.42s raspberrypi : Test for raspberry pi /proc/cpuinfo -------------------------------------------------------------------------------------------------------------------------- 0.36s k3s/master : Register node-token file access mode -------------------------------------------------------------------------------------------------------------------------- 0.33s prereq : Enable IPv4 forwarding -------------------------------------------------------------------------------------------------------------------------------------------- 0.32s k3s/master : Create directory .kube ---------------------------------------------------------------------------------------------------------------------------------------- 0.31s k3s/master : Read node-token from master ----------------------------------------------------------------------------------------------------------------------------------- 0.30s raspberrypi : Test for raspberry pi /proc/device-tree/model ---------------------------------------------------------------------------------------------------------------- 0.26s k3s/master : Restore node-token file access -------------------------------------------------------------------------------------------------------------------------------- 0.23s k3s/master : Copy config file to user home directory ----------------------------------------------------------------------------------------------------------------------- 0.21s k3s/master : Create crictl symlink ----------------------------------------------------------------------------------------------------------------------------------------- 0.21s

Plus there is a pretty good ansible playbook https://github.com/k3s-io/k3s-ansible

We have develop ansible role to do that. We did not put it as opensource yet. But there is one in the comunity which looks good: https://github.com/k3s-io/k3s-ansible