Nomad VS k3s

In previous post, we discussed creating a basic Nomad cluster in the Vultr cloud. Here, we will use the cluster created to deploy a load-balanced sample web app using the service discovery capability of Nomad and its native integration with the Traefik load balancer. The source code is available here for the reference.

Nomad is really awesome!

I only heard of this today, but it looks really interesting. It seems to finally get Kubernetes a bit closer to something like https://www.nomadproject.io/ in terms of complexity to install and operate.

In the rapidly evolving landscape of container orchestration and management, two prominent players have emerged: Kubernetes and HashiCorp's Nomad. While Kubernetes has gained widespread adoption and popularity, Nomad provides a compelling alternative that stands out for its simplicity and efficiency. In this blog post, we'll explore the advantages of using Nomad over Kubernetes and why it might be the right choice for certain use cases.

I can't discern how many are just those "dependabot" bumps but the 1400 forks show some are active https://github.com/hashicorp/nomad/forks?include=active&page... including CircleCI who I would think have a stake in a libre Nomad https://github.com/circleci/nomad/tree/circleci/release-1.5....

Now maybe their goals don't align with the community, and/or they don't want to be in the maintainer business for such a project, but better than nothing

Could this be a solution? nomad

It worth noting that Nomad UI(a official web admin panel) has log tailing utility built-in so maybe partial work has already been done. The developers may have other concerns.

The related issue is https://github.com/hashicorp/nomad/issues/10220

While I do understand the reasoning in their FAQ on the subject (https://www.hashicorp.com/license-faq). I however failed to noticed those intentions in their license text (https://github.com/hashicorp/nomad/commit/b3e30b1dfa185d9437...).

Specifically the part in FAQ which says "internal production use is fine", but then license says that "non-production use only" and then "You may make production use of the Licensed Work, provided such use does not include offering the Licensed Work to third parties on a hosted or embedded basis which is competitive with HashiCorp's products.".

IANAL, but even to me this statement is full loopholes. WHO do we consider 3rd party? WHAT do we consider "hosted or embedded basis"? WHEN do we consider it "competitive with Hashicorps products"?

My repeated tests show that when an allocation that is using a volume is deallocated, nomad doesn't release the volume. So, it is inaccessable to the next allocation.

For self-hosting I've found https://k3s.io to be really good from the SUSE people. Works on basically any Linux distro and makes self-hosting k8s not miserable.

Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319

For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.

We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.

K3S: is a lightweight distribution of Kubernetes that is designed for resource-constrained environments. It is an excellent option for running Kubernetes on a virtual machine or cloud server.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Kubernetes - k3s distribution

I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.

You should be familiar with Kubernetes and have set up a Kubernetes cluster. I recommend k3s.

K3s' go.mod[0] is insane.

[0] https://github.com/k3s-io/k3s/blob/master/go.mod