Our great sponsors
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
nerdctl
contaiNERD CTL - Docker-compatible CLI for containerd, with support for Compose, Rootless, eStargz, OCIcrypt, IPFS, ... (by AkihiroSuda)
-
-
Fly CDN
Discontinued A set of useful libraries for Edge Apps. Run locally, write tests, and integrate it into your deployment process. Move fast and maybe don't break things? Because, gosh darnit, you're an adult. (by superfly)
Author (https://github.com/Roblox/nomad-driver-containerd) here.
Not really, Podman and containerd are two different technologies, although both allow you to move away from Docker for various reasons (smaller CPU, memory footprint, better security etc). If you are invested into Red Hat container stack, podman makes more sense. However containerd is more universal.
K8s is already moving away from docker, and directly into containerd. Most recently they deprecated dockershim, and users now need to switch to containerd (since docker also uses containerd under the hood, and it doesn't make sense for the orchestration system to run a monolithic service like docker where it just need to launch the workloads)
Some reference links of k8s or PaaS build on top of k8s moving to containerd
Coincidentally my home server is also named Hydra, running Nomad on CoreOS/Flatcar. I've had this setup for years now running without issue (unless I managed to mess it up myself by, for example, changing the IP and not following proper procedures to restore the Raft cluster).
Recently I deployed K3s on the same node to add some new workloads, but now that I want to move those workloads to Nomad to get rid of the cpu and memory usage of K8s. I'm running into what is becoming my main problem with Nomad that I never thought I had.
With all it's complexities, getting something to run on K8s is as simple as adding a Helm chart name to a Terraform config file and running apply. Maybe I need to set a value or volume, but that's it. Everything below is pretty much standardised.
With Nomad however, it's benefit of doing only one thing very well also means that all the other things like ingress and networking need to be figured out yourself. And since there is no standard regarding these everyone invents their own, preventing something like k8s-at-home [0] to emerge. Also K8s is pretty agnostic in container backend where Nomad needs configurations for every driver.
I think writing your own Helm charts for everything would suck more than writing the Nomad configs. Though a lot could be automatically generated for both of them. But I'm missing a community repository of sorts for Nomad.
[0] https://github.com/k8s-at-home/charts
I myself would love to see more usage of HCL... after writing a lot of Terraform configurations, it feels so much nicer for me than any JSON or YAML/Helm configuration I have written to this day. We should agree on some kind of HCL-based industry standard and leave all these workarounds via JSON/YAML behind us... e.g. doing Helm logic with these Go templates just looks like a step backwards after writing Terraform HCL code for the last 2-3 years.
I understand JSON as a simple interchange format between systems, and is here to stay, but I don't understand all this YAML stuff, with all its quirks, from the K8s/DevOps people, when we have the much nicer HCL...
For anyone not used to HCL: https://github.com/hashicorp/hcl
I've been there. You basically want to be able `cd` into vault and list the contents interactively, but you can't.
While the Web UI is probably the best vault explorer available, you might want to take a look at Vaku[1].
[1]: https://github.com/lingrino/vaku/blob/main/docs/cli/vaku.md#...
A whole lot of "you most likely don't need them and the increase in complexity makes it a pain to maintain" and "Kubernetes is exceedingly complex" in this thread.
I agree that you probably don't need Kubernetes, and perhaps yeah it could be considered complex.
But I think it's the right fit for most developers. Kubernetes is not Kubernetes. Kubernetes is some base machinery, yes, but it's also a pattern, for writing controllers/operators that take Kubernetes Objects and turn them into things. Take a Postgres object and turn it into a running, healing, backing-up replicated postgres cluster. Take a SQS object and with ACK turn it into a real SQS.
Kubernetes & cloud native in general proposes that you should have working models for the state of your world. In addition to the out-of-the-box machinery you get for running containers (deployment-sets), exposing them (services), &c, you get this pattern. You get other folks building operators/controllers that implement this pattern[1]. You get a consistent, powerful, extensible way of building.
Nothing else comes close. There's nothing remotely as interesting in the field right now. The Cult of Easy is loud & bitterly angry about Kubernetes, hates it's "complexity", but what is actually complex is having a dozen different operational environments for different tools & systems. What is actually complex is operating systems yourself, rather than having operators to maintain systems. Kubernetes has some initial costs, it can feel daunting, but it is radically simpler in the long run because _it has a paradigm,_ an all inclusive paradigm that all systems can fit into, and the autonomic behaviors this paradigm supports radically transfer operational complexity from human to computer, across that broad/all-inclusive range of systems.
There's a lot of easier this/harder that. No one tries to pitch Nomad or anything else as better, as deeper, as being more consistent, having a stronger core. Every article you hear on an alternative to Kubernetes is 98% "this was easier". I think those people, largely, miss the long game, the long view. A system that can adapt, that operationally can serve bigger & bigger scopes, ought to pay dividends to you as years go by. Kubernetes may take you longer to get going. But it is time enormously well spent, that will increase your capability & mastery of the world, & bring you together with others building radically great systems whether at home[2][3] or afar. It will be not just a way of running infrastructure, but help you re-think how you develop, and how to expose your own infrastructure & ideas more consistently, more clearly, in the new pattern language of autonomic machines that we have only just begun to build together.
I encourage the bold explorers out there, learn Kubernetes, run Kubernetes. And to those of you pitching other things, please, I want you to talk up your big game better, tell me late-game scenarios, tell me how your system & I are going to grow together, advance each other.
[1] https://kubernetes.io/docs/concepts/architecture/controller/...
[2] https://github.com/onedr0p/home-cluster
[3] https://github.com/k8s-at-home/awesome-home-kubernetes
consul connect handles this, how you get traffic to the ingresses is still DIY... kinda. you can also use consul catalog + traefik (I've actually put in some PRs myself to make traefik work with a really huge consul catalog so you can scale it to fronting thousands of services at once). there's also fabio. you can also get bgp ip injection with consul via https://github.com/mayuresh82/gocast run as a system job to get traffic to any LB (or any workload) if that's an option.
> storage provider
One area, where containerd didn't had a first class support was CLI. the default containerd CLI "ctr" has a very naive implementation. The reason for that I believe is, containerd as a system was never meant to be consumed by humans, and was designed to be consumed by higher layers e.g. orchestration systems like nomad or k8s. However, with the deprecation of dockershim in k8s, and users moving to containerd, a new docker compatible CLI came out:
https://github.com/AkihiroSuda/nerdctl
If you just have containerd running on your system (with no docker daemon running), you can just install nerdctl and add
alias docker="nerdctl"
to your ~/.bashrc file.
Then you can just run any docker commands the way you used to with docker, and it will run those commands against the containerd API giving you the same CLI experience that you used to have with docker.
> If you want Kubernetes it's probably a better idea to use a hosted Kubernetes solution like Google's offering.
This may be not true in the future with distributions like k0s[1]
[1]: https://k0sproject.io/
I think we have passed the first stage where people have gone through a couple of disasters and have learned they actually didn't need majority of the features k8s offers at their scale. They are now actively looking for simpler tools which opens space for nomad and co. Plus success stories from companies like https://fly.io(yes I like them) with nomad are pilling up.