Nomad VS k3s

Must be hashicorp's nomad: https://www.nomadproject.io/

That's a false statement as far as the technical aspects are concerned (Swarm is still usable and supported), but is a true statement when you look at the social aspects (Kubernetes won the container wars and now even Nomad is uncommon to run into).

Right now the company i'm in uses Swarm in a lot of places due to its simplicity (Compose file support) and low resource usage - Swarm hits the sweet spot when it comes to getting started with container orchestration and doing so without needing multiple people to wrangle the technical complexity of Kubernetes, or large VMs to deal with its resource usage, at least in on prem environments.

In combination with Portainer (https://www.portainer.io/) it's perhaps one of the best ways to get things done, when you expect everything to just work and aren't doing something too advanced (think along the lines of 10 servers, rather than 100, which is probably most of the deployments out there).

I actually wrote about some of its advantages in my blog post, "Docker Swarm over Kubernetes": https://blog.kronis.dev/articles/docker-swarm-over-kubernete...

That said, if there are any good options to replace Swarm, it has to either be Hashicorp Nomad (https://www.nomadproject.io/) which is a really nice platform, especially when coupled with Consul (https://www.consul.io/), as long as you can get past the weirdness of HCL. Alternatively, it has to be K3s (https://k3s.io/), which gives you Kubernetes without the insane bloat and hardware usage.

I actually benchmarked K3s against Docker Swarm in similar app deployments: 1 leader server, 2 follower servers, running a Ruby on Rails app and an ingress, while they're under load testing by K6 (https://k6.io/). I was attempting to see whether COVID contract tracking with GPS would be viable as far as the system load goes in languages with high abstraction level, here's more info about that: https://blog.kronis.dev/articles/covid-19-contact-tracing-wi...

Honestly, the results were pretty close - on the follower servers, the overhead of the orchestrator agents were a few percent (K3s being heavier, but a few dozen MB here or there not being too relevant), whereas the bigger differences were in the leader components, where K3s was heavier almost by a factor of two, which isn't too much when you consider how lightweight Swarm is (there was a difference of a few hundred MB) and the CPU usage was reasonably close in both of the cases as well. Sadly, the text of the paper is in Latvian, so it's probably of no use to anyone, but i advise you to do your own benchmarks! Being a student, i couldn't afford many servers then, so it's probably a good idea to benchmark those with more servers.

Of note, on those VPSes (4 GB of RAM, single core), the full Kubernetes wouldn't even start, whereas at work, trying to get the resources for also running Rancher on top of a "full" Kubernetes cluster (e.g. RKE) can also take needlessly long due to the backlash from ops. Also, personally i find the Compose syntax to be far easier to deal with, rather than the amalgamation that Kubernetes uses, Helm probably shouldn't even be a thing if the deployment descriptors weren't so bloated. Just look at this: https://docs.docker.com/compose/compose-file/compose-file-v3...

In short:

  - Docker Swarm is pretty good when you're starting out with containers and is reasonably stable and easy to use

Also if you don’t want to look at kube then check out Nomad. https://www.nomadproject.io/

Hahaha sure: https://www.nomadproject.io it’s by HashiCorp.

Yeah, this pains me too. Here's a relevant issue to keep an eye on:

https://github.com/hashicorp/nomad/issues/3854

I've used an nginx-based S3 proxy in the past to get around this. Not ideal but it works.

At Lob, we currently use Convox as our deployment platform, a “roll your own Platform-as-a-Service” that you can install to handle container orchestration on AWS’s ECS (Elastic Container Service). Convox is showing its age and this year we began the process of replacing Convox with HashiCorp’s Nomad, a flexible workload orchestrator to deploy and manage our containers on AWS.

I'm running 2.00.13 using the official Docker image. If it makes any difference, I'm running this in a K3s cluster (lightweight Kubernetes), itself running on Ubuntu 20.04.3 servers.

Here is link number 1 - Previous text "K3s"

If you only have a few nodes and are on-prem, you might look into K3s and/or K3OS. It tries to be a "low resource" kubernetes, and doesn't assume isolated master nodes.

Everything runs fine on model B with one 700MHz core and 512MB of ram. Checkout the raspberry-config repo for more details and updates. However I can't really do much with this hardware. That's why as next steps I'm considering buying more RBPs and setting up a clustered environment with k3s.

This mirrors my experience with K8s, though one can suggest that the software running on the cluster also played a part in it.

Have you looked into K3s or other lightweight distros of Kubernetes, though? https://k3s.io/

That's what i turned to when i needed to learn the Kubernetes API and explore running clusters, but didn't like my small VPSes (~4 GB of RAM) failing to launch anything because of the frankly insane resource usage of full K8s clusters.

Docs: https://k3s.io

Great example. Just to elaborate a little more. https://k3s.io/ is kubernetes but packed into a binary. It works across multiple platforms and even raspberry pis. It's footprint in compared to k8s is significant. K3s 0.25cpu and 512mb ram. It almost contains everything k8s, but missing some alpha and advanced CRD capability.

That's a false statement as far as the technical aspects are concerned (Swarm is still usable and supported), but is a true statement when you look at the social aspects (Kubernetes won the container wars and now even Nomad is uncommon to run into).

Right now the company i'm in uses Swarm in a lot of places due to its simplicity (Compose file support) and low resource usage - Swarm hits the sweet spot when it comes to getting started with container orchestration and doing so without needing multiple people to wrangle the technical complexity of Kubernetes, or large VMs to deal with its resource usage, at least in on prem environments.

In combination with Portainer (https://www.portainer.io/) it's perhaps one of the best ways to get things done, when you expect everything to just work and aren't doing something too advanced (think along the lines of 10 servers, rather than 100, which is probably most of the deployments out there).

I actually wrote about some of its advantages in my blog post, "Docker Swarm over Kubernetes": https://blog.kronis.dev/articles/docker-swarm-over-kubernete...

That said, if there are any good options to replace Swarm, it has to either be Hashicorp Nomad (https://www.nomadproject.io/) which is a really nice platform, especially when coupled with Consul (https://www.consul.io/), as long as you can get past the weirdness of HCL. Alternatively, it has to be K3s (https://k3s.io/), which gives you Kubernetes without the insane bloat and hardware usage.

I actually benchmarked K3s against Docker Swarm in similar app deployments: 1 leader server, 2 follower servers, running a Ruby on Rails app and an ingress, while they're under load testing by K6 (https://k6.io/). I was attempting to see whether COVID contract tracking with GPS would be viable as far as the system load goes in languages with high abstraction level, here's more info about that: https://blog.kronis.dev/articles/covid-19-contact-tracing-wi...

Honestly, the results were pretty close - on the follower servers, the overhead of the orchestrator agents were a few percent (K3s being heavier, but a few dozen MB here or there not being too relevant), whereas the bigger differences were in the leader components, where K3s was heavier almost by a factor of two, which isn't too much when you consider how lightweight Swarm is (there was a difference of a few hundred MB) and the CPU usage was reasonably close in both of the cases as well. Sadly, the text of the paper is in Latvian, so it's probably of no use to anyone, but i advise you to do your own benchmarks! Being a student, i couldn't afford many servers then, so it's probably a good idea to benchmark those with more servers.

Of note, on those VPSes (4 GB of RAM, single core), the full Kubernetes wouldn't even start, whereas at work, trying to get the resources for also running Rancher on top of a "full" Kubernetes cluster (e.g. RKE) can also take needlessly long due to the backlash from ops. Also, personally i find the Compose syntax to be far easier to deal with, rather than the amalgamation that Kubernetes uses, Helm probably shouldn't even be a thing if the deployment descriptors weren't so bloated. Just look at this: https://docs.docker.com/compose/compose-file/compose-file-v3...

In short:

  - Docker Swarm is pretty good when you're starting out with containers and is reasonably stable and easy to use

K3s - a lightweight, highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances.