Kubernetes at Home with K3s

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • SonarQube - Static code analysis for 29 languages.
  • Scout APM - Less time debugging, more time building
  • SaaSHub - Software Alternatives and Reviews
  • Portainer

    Making Docker and Kubernetes management easy.

    That's a false statement as far as the technical aspects are concerned (Swarm is still usable and supported), but is a true statement when you look at the social aspects (Kubernetes won the container wars and now even Nomad is uncommon to run into).

    Right now the company i'm in uses Swarm in a lot of places due to its simplicity (Compose file support) and low resource usage - Swarm hits the sweet spot when it comes to getting started with container orchestration and doing so without needing multiple people to wrangle the technical complexity of Kubernetes, or large VMs to deal with its resource usage, at least in on prem environments.

    In combination with Portainer (https://www.portainer.io/) it's perhaps one of the best ways to get things done, when you expect everything to just work and aren't doing something too advanced (think along the lines of 10 servers, rather than 100, which is probably most of the deployments out there).

    I actually wrote about some of its advantages in my blog post, "Docker Swarm over Kubernetes": https://blog.kronis.dev/articles/docker-swarm-over-kubernete...

    That said, if there are any good options to replace Swarm, it has to either be Hashicorp Nomad (https://www.nomadproject.io/) which is a really nice platform, especially when coupled with Consul (https://www.consul.io/), as long as you can get past the weirdness of HCL. Alternatively, it has to be K3s (https://k3s.io/), which gives you Kubernetes without the insane bloat and hardware usage.

    I actually benchmarked K3s against Docker Swarm in similar app deployments: 1 leader server, 2 follower servers, running a Ruby on Rails app and an ingress, while they're under load testing by K6 (https://k6.io/). I was attempting to see whether COVID contract tracking with GPS would be viable as far as the system load goes in languages with high abstraction level, here's more info about that: https://blog.kronis.dev/articles/covid-19-contact-tracing-wi...

    Honestly, the results were pretty close - on the follower servers, the overhead of the orchestrator agents were a few percent (K3s being heavier, but a few dozen MB here or there not being too relevant), whereas the bigger differences were in the leader components, where K3s was heavier almost by a factor of two, which isn't too much when you consider how lightweight Swarm is (there was a difference of a few hundred MB) and the CPU usage was reasonably close in both of the cases as well. Sadly, the text of the paper is in Latvian, so it's probably of no use to anyone, but i advise you to do your own benchmarks! Being a student, i couldn't afford many servers then, so it's probably a good idea to benchmark those with more servers.

    Of note, on those VPSes (4 GB of RAM, single core), the full Kubernetes wouldn't even start, whereas at work, trying to get the resources for also running Rancher on top of a "full" Kubernetes cluster (e.g. RKE) can also take needlessly long due to the backlash from ops. Also, personally i find the Compose syntax to be far easier to deal with, rather than the amalgamation that Kubernetes uses, Helm probably shouldn't even be a thing if the deployment descriptors weren't so bloated. Just look at this: https://docs.docker.com/compose/compose-file/compose-file-v3...

    In short:

      - Docker Swarm is pretty good when you're starting out with containers and is reasonably stable and easy to use

  • Nomad

    Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

    That's a false statement as far as the technical aspects are concerned (Swarm is still usable and supported), but is a true statement when you look at the social aspects (Kubernetes won the container wars and now even Nomad is uncommon to run into).

    Right now the company i'm in uses Swarm in a lot of places due to its simplicity (Compose file support) and low resource usage - Swarm hits the sweet spot when it comes to getting started with container orchestration and doing so without needing multiple people to wrangle the technical complexity of Kubernetes, or large VMs to deal with its resource usage, at least in on prem environments.

    In combination with Portainer (https://www.portainer.io/) it's perhaps one of the best ways to get things done, when you expect everything to just work and aren't doing something too advanced (think along the lines of 10 servers, rather than 100, which is probably most of the deployments out there).

    I actually wrote about some of its advantages in my blog post, "Docker Swarm over Kubernetes": https://blog.kronis.dev/articles/docker-swarm-over-kubernete...

    That said, if there are any good options to replace Swarm, it has to either be Hashicorp Nomad (https://www.nomadproject.io/) which is a really nice platform, especially when coupled with Consul (https://www.consul.io/), as long as you can get past the weirdness of HCL. Alternatively, it has to be K3s (https://k3s.io/), which gives you Kubernetes without the insane bloat and hardware usage.

    I actually benchmarked K3s against Docker Swarm in similar app deployments: 1 leader server, 2 follower servers, running a Ruby on Rails app and an ingress, while they're under load testing by K6 (https://k6.io/). I was attempting to see whether COVID contract tracking with GPS would be viable as far as the system load goes in languages with high abstraction level, here's more info about that: https://blog.kronis.dev/articles/covid-19-contact-tracing-wi...

    Honestly, the results were pretty close - on the follower servers, the overhead of the orchestrator agents were a few percent (K3s being heavier, but a few dozen MB here or there not being too relevant), whereas the bigger differences were in the leader components, where K3s was heavier almost by a factor of two, which isn't too much when you consider how lightweight Swarm is (there was a difference of a few hundred MB) and the CPU usage was reasonably close in both of the cases as well. Sadly, the text of the paper is in Latvian, so it's probably of no use to anyone, but i advise you to do your own benchmarks! Being a student, i couldn't afford many servers then, so it's probably a good idea to benchmark those with more servers.

    Of note, on those VPSes (4 GB of RAM, single core), the full Kubernetes wouldn't even start, whereas at work, trying to get the resources for also running Rancher on top of a "full" Kubernetes cluster (e.g. RKE) can also take needlessly long due to the backlash from ops. Also, personally i find the Compose syntax to be far easier to deal with, rather than the amalgamation that Kubernetes uses, Helm probably shouldn't even be a thing if the deployment descriptors weren't so bloated. Just look at this: https://docs.docker.com/compose/compose-file/compose-file-v3...

    In short:

      - Docker Swarm is pretty good when you're starting out with containers and is reasonably stable and easy to use

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • consul

    Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

    That's a false statement as far as the technical aspects are concerned (Swarm is still usable and supported), but is a true statement when you look at the social aspects (Kubernetes won the container wars and now even Nomad is uncommon to run into).

    Right now the company i'm in uses Swarm in a lot of places due to its simplicity (Compose file support) and low resource usage - Swarm hits the sweet spot when it comes to getting started with container orchestration and doing so without needing multiple people to wrangle the technical complexity of Kubernetes, or large VMs to deal with its resource usage, at least in on prem environments.

    In combination with Portainer (https://www.portainer.io/) it's perhaps one of the best ways to get things done, when you expect everything to just work and aren't doing something too advanced (think along the lines of 10 servers, rather than 100, which is probably most of the deployments out there).

    I actually wrote about some of its advantages in my blog post, "Docker Swarm over Kubernetes": https://blog.kronis.dev/articles/docker-swarm-over-kubernete...

    That said, if there are any good options to replace Swarm, it has to either be Hashicorp Nomad (https://www.nomadproject.io/) which is a really nice platform, especially when coupled with Consul (https://www.consul.io/), as long as you can get past the weirdness of HCL. Alternatively, it has to be K3s (https://k3s.io/), which gives you Kubernetes without the insane bloat and hardware usage.

    I actually benchmarked K3s against Docker Swarm in similar app deployments: 1 leader server, 2 follower servers, running a Ruby on Rails app and an ingress, while they're under load testing by K6 (https://k6.io/). I was attempting to see whether COVID contract tracking with GPS would be viable as far as the system load goes in languages with high abstraction level, here's more info about that: https://blog.kronis.dev/articles/covid-19-contact-tracing-wi...

    Honestly, the results were pretty close - on the follower servers, the overhead of the orchestrator agents were a few percent (K3s being heavier, but a few dozen MB here or there not being too relevant), whereas the bigger differences were in the leader components, where K3s was heavier almost by a factor of two, which isn't too much when you consider how lightweight Swarm is (there was a difference of a few hundred MB) and the CPU usage was reasonably close in both of the cases as well. Sadly, the text of the paper is in Latvian, so it's probably of no use to anyone, but i advise you to do your own benchmarks! Being a student, i couldn't afford many servers then, so it's probably a good idea to benchmark those with more servers.

    Of note, on those VPSes (4 GB of RAM, single core), the full Kubernetes wouldn't even start, whereas at work, trying to get the resources for also running Rancher on top of a "full" Kubernetes cluster (e.g. RKE) can also take needlessly long due to the backlash from ops. Also, personally i find the Compose syntax to be far easier to deal with, rather than the amalgamation that Kubernetes uses, Helm probably shouldn't even be a thing if the deployment descriptors weren't so bloated. Just look at this: https://docs.docker.com/compose/compose-file/compose-file-v3...

    In short:

      - Docker Swarm is pretty good when you're starting out with containers and is reasonably stable and easy to use

  • k3s

    Lightweight Kubernetes

    That's a false statement as far as the technical aspects are concerned (Swarm is still usable and supported), but is a true statement when you look at the social aspects (Kubernetes won the container wars and now even Nomad is uncommon to run into).

    Right now the company i'm in uses Swarm in a lot of places due to its simplicity (Compose file support) and low resource usage - Swarm hits the sweet spot when it comes to getting started with container orchestration and doing so without needing multiple people to wrangle the technical complexity of Kubernetes, or large VMs to deal with its resource usage, at least in on prem environments.

    In combination with Portainer (https://www.portainer.io/) it's perhaps one of the best ways to get things done, when you expect everything to just work and aren't doing something too advanced (think along the lines of 10 servers, rather than 100, which is probably most of the deployments out there).

    I actually wrote about some of its advantages in my blog post, "Docker Swarm over Kubernetes": https://blog.kronis.dev/articles/docker-swarm-over-kubernete...

    That said, if there are any good options to replace Swarm, it has to either be Hashicorp Nomad (https://www.nomadproject.io/) which is a really nice platform, especially when coupled with Consul (https://www.consul.io/), as long as you can get past the weirdness of HCL. Alternatively, it has to be K3s (https://k3s.io/), which gives you Kubernetes without the insane bloat and hardware usage.

    I actually benchmarked K3s against Docker Swarm in similar app deployments: 1 leader server, 2 follower servers, running a Ruby on Rails app and an ingress, while they're under load testing by K6 (https://k6.io/). I was attempting to see whether COVID contract tracking with GPS would be viable as far as the system load goes in languages with high abstraction level, here's more info about that: https://blog.kronis.dev/articles/covid-19-contact-tracing-wi...

    Honestly, the results were pretty close - on the follower servers, the overhead of the orchestrator agents were a few percent (K3s being heavier, but a few dozen MB here or there not being too relevant), whereas the bigger differences were in the leader components, where K3s was heavier almost by a factor of two, which isn't too much when you consider how lightweight Swarm is (there was a difference of a few hundred MB) and the CPU usage was reasonably close in both of the cases as well. Sadly, the text of the paper is in Latvian, so it's probably of no use to anyone, but i advise you to do your own benchmarks! Being a student, i couldn't afford many servers then, so it's probably a good idea to benchmark those with more servers.

    Of note, on those VPSes (4 GB of RAM, single core), the full Kubernetes wouldn't even start, whereas at work, trying to get the resources for also running Rancher on top of a "full" Kubernetes cluster (e.g. RKE) can also take needlessly long due to the backlash from ops. Also, personally i find the Compose syntax to be far easier to deal with, rather than the amalgamation that Kubernetes uses, Helm probably shouldn't even be a thing if the deployment descriptors weren't so bloated. Just look at this: https://docs.docker.com/compose/compose-file/compose-file-v3...

    In short:

      - Docker Swarm is pretty good when you're starting out with containers and is reasonably stable and easy to use

  • k3d

    Little helper to run CNCF's k3s in Docker

    Only on mac though, because docker has extra overhead due to running on a vm there. I run kubernetes all the time in my linux desktop and it consume <5% cpu on idle, and that's with more than a dozen pods running in the background (postgres, postgis, mysql, redis, memcached, a bunch of django webapps, some test wordpress instances, multiple, and a pihole instance).

    I got fed up with the performance on mac and tried using https://github.com/rancher/k3d instead, which seems to cut the idle cpu usage by half.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts