Vagrant VS Packer

Ensuring high availability with limited resources can be challenging. I recently want to proove you can do it using Keepalived and just two servers 💪✨. To prove it, I used Vagrant. Here's a quick rundown of my journey! 🚀

Vagrant - Make sure you have Vagrant installed on your machine. You can download it from Vagrant's official site.

I do believe that using containers makes a ton of sense in writing data pipelines. You can use the same image to develop and run the pipeline, preventing "it works on my machine" issues. You can test different variations of the image without having to stand up additional infrastructure or potentially breaking the workflows of others who're using the same infrastructure. Finally, knowledge of containerization is increasingly expected of all engineers, while knowledge of other tools that solve similar issues (like Vagrant or Ansible) is less common.

Running NixOS on a virtual machine (VM) is a safe and reproducible way to test such configurations. As for VMs, I have used VirtualBox, Vagrant and lxd in the past. However, I have found QEMU to be the simplest and most flexible solution for my needs.

I found out that the error was that initialy I was using an old version of the vagrant utility for vmware: 1.0.21, so I read this post: https://github.com/hashicorp/vagrant/issues/12052 and there I saw that I needed to install the new version, in my case 1.0.22 which I mentioned on the point #3.

Vagrant is a tool for building and managing virtual machine environments in a single workflow. Developed by Hashicorp, it is used to replicate multiple virtual environments. It can efficiently run in all virtualized environments, providing the highest level of isolation to users.

Another important tool from the same organization is Vagrant, which provides extra help in running VMs built with Packer. Of course, the choice of a VM provider is also very important, as some VM providers may not be supported on certain platforms. For example, there are no VMware or VirtualBox releases that support Apple Silicon. However, QEMU is supported on most platforms, including Apple Silicon, which is why this provider was chosen here.

https://github.com/hashicorp/vagrant/issues/1052#issuecommen...

Again at this point the jokes are frankly writing themselves. Like bro make it possible for people to follow your advice.

Also if you directly state or indirectly insinuate that your tool is ANY/ALL OF Local First, or Open Source, or Free As In Freedom you better have offline docs.

If you don't have offline docs your users and collaborators don't have Freedom 1. If you can't exercise Freedom 1 you are severely hampered in your ability to exercise Freedoms 0, 2, or 3 for any nontrivial FOSS system.

The problem has gotten so bad the I started the Freedom Respecting Technology movement which I'm gonna plug here: https://makesourcenotcode.github.io/freedom_respecting_techn...

Vagrant

For building, I have decided to use packer since it has a lot of examples available online for building common images. However, I don't like the hcl file format due to the limited amount of tooling available for it, so I have decided to use the json configuration syntax. Since I like yaml more than json and since packer doesn't support it natively, the program firstly converts the yaml to a temporary json and then executes packer.

> But then Packer has no concept of multi-stage or copying in assets from other containers.

I don't know about containers specifically, since I've never bothered to use packer for that process, but it does seem that packer supports multi-step artifact production and their example even uses docker to demonstrate it https://github.com/hashicorp/packer/blob/v1.9.5/website/cont...

So performance cannot be said to be better with Docker, why choose Docker then? Better reasons are that you can strip down a Docker image much easier than an OS. This is critical for us due to security requirements. While Python requires a lot of OS features, the majority of the OS is still bloat. Every piece of bloat is a potential attack vector (each of these unused components might have one or more CVEs that we need to patch, even though we don't even use that software). Another reason is that the build process of Docker is much simpler to manage. There are tools such as Packer that allow similar processes for VMs, but these are not as standardized as the open container initiative (OCI - which Docker adheres to).

Setting up the VM and all the necessary tools usually takes time and effort. Automating this process would be much faster, more convenient, and significantly less error-prone. While one can write scripts to set up VMs, this approach requires new implementations for each virtualization software technology. Various tools exist for this purpose, but I am going to use Packer because it is open source, widely adopted, and well-supported. It supports all modern VM providers, such as VirtualBox, VMware, KVM, and various cloud providers. It is also highly configurable and can be extended if you need functionality not yet supported by the tool.

The missing piece of puzzle is the AMI "golden image" that will be used to start the instances in autoscaling group. The AMI has to have NGINX and PHP installed with the list of required modules enabled. The great tool to brew one is hashicorp packer.

To manage a VM, you can use something as simple as just manual actions over SSH, or can use tools like Ansible, Hashicorp's Packer and Terraform or other automations. For an app where there is minimal load and security/reliability concern, VMs are still a great option that provide a lot of value for the buck

Server templating: Using Packer has never been easier to create reusable server configurations in a platform-independent and documented manner.

I'll give you hard, but rewarding and easy to modify(once you know what you're doing) way. Packer may be a thing you're looking for.

It is straightforward to build them with packer (I have built AMIs for 13.0 and 13.1, but 13.2 should be exactly the same). I've been meaning to write a blog post about it for a while, but have not gotten to it yet... In any case, what I am doing is using the EBS Surrogate Builder to start an instance running the official FreeBSD 13.2 image with an extra volume attached and run a script to create a zpool on the extra volume and bootstrap and configure FreeBSD 13.2-RELEASE on it. After that packer takes care of creating an AMI out of that extra volume, so you can use it... If you have any issues, let me know, and maybe I will finally get to writing that blog post...

HashiCorp Packer is a tool for creating machine images for a variety of platforms, including AWS, Azure, and VMware. It allows you to define machine images as code and supports a wide range of configuration options.