k8s-lab-terraform-libvirt VS libguestfs

https://blog.ruanbekker.com/blog/2020/10/08/using-the-libvir...

I include the blog link because there is some nuance in how to get the path right for community Terraform providers that aren't in the Hashicorp registry. The documentation on the GitHub project isn't quite up to date with respect to how the latest versions of Terraform expect the plugin paths to be set up.

I've done this pretty successfully with all the major Linux distros minus Arch, which requires some bootstrapping to get an iso that Packer can work with (no such thing as an answers file for Arch). It's not that big a deal, though. Just find some instructions on how to create and mount a cloud-init iso in addition to the installer iso and use that to add an ssh public key so you can script the installation steps externally. I actually think Packer can do this, but I just haven't gotten it to work yet and have relied on shell scripts.

Hyper-V actually has a very comprehensive PowerShell module that is pretty well documented, by the way: https://docs.microsoft.com/en-us/powershell/module/hyper-v/?.... I've found it pretty easy to use and actually got the Arch auto-provision working on Hyper-V in Windows before I got it working in KVM in Linux.

Another thing is you can just use the cloud images and cloud-init for bootstrapping everything pretty easily, even on-prem. cloud-init has a "no cloud" config option, as mentioned above, where you just mount an iso with the config data as a DVD drive and cloud-init will find it automatically when the distro iso boots.

This guy has a pretty comprehensive example of how to set up a kubernetes homelab entirely using the libvirt Terraform provider from Ubuntu cloud images bootstrapped with cloud-init: https://github.com/zloeber/k8s-lab-terraform-libvirt

We started off doing this, but you end up with enormous diffs which are themselves confusing. Example, only about 5% of this change is non-generated:

https://github.com/libguestfs/libguestfs/commit/5186251f8f68...

And inside the registry. The apparently correct way to distinguish them is using the build ID:

https://github.com/libguestfs/libguestfs/commit/824c74574893...

NDB works great but another option is libguestfs. https://libguestfs.org/

Have a look here

Consider using libguestfs.

There's a lot of useful command-line tooling for KVM and QEMU-based virt. Here's a small selection of useful tools:

• virsh — This[1] is libvirt's shell interface; and gives you access to the rich set of libvirt APIs.

• virt-builder — Use this for rapidly building minimal or customized virtual machines; it's greatly flexible; check out its man page[2]. And here's[3] a quick example that connects both virt-builder and virsh together.

• virt-install — Use this if you don't like the default build of the template images from virt-builder; it lets you create "headless" servers via 'kickstart' and Linux OS trees from the command-line.

• guestfish and libguestfs suite[4] — This rich set of tools help you in a variety of use-cases: repairing your broken disk images, editing, cloning, debugging disk images, and more. It has saved my behind a lot of times.

• qemu-img[5] – This Swiss Army knife lets you powerfully manipulate disk images (QCOW2, raw, et al) offline. Example operations include: create images, backing chains, offline snapshots, disk image merging, and convert disk images from one format to another, and more.

[1] https://libvirt.org/manpages/virsh.html

[2] https://libguestfs.org/virt-builder.1.html

[3] https://developer.fedoraproject.org/tools/virt-builder/about...

[4] http://libguestfs.org/

[5] https://qemu.readthedocs.io/en/latest/tools/qemu-img.html

That's a good point. As long as the project has a setup.py or pyproject.toml available, it can usually be installed from the repo. For libguestfs it looks like they do some pre-processing on their setup.py so that wouldn't work, it's lucky that they had this alternative set up already. :)

Nah, this virt-install preseed script is faster, or even just run virt-builder debian-10 and they're both libvirt not hacky qemu scripts