design VS firebuild

> we'd have been fighting cni complexity to make it work.

Appreciate the candid responses, thanks for taking the time. That ipv6 wireguard peering post was really fascinating I read that too. Wireguard has been quite the a game-changer in it's space as well and a lot of value IMO is just in the simplicity and difficulty of misconfiguration, even though the performance is also fantastic.

Grateful that ya'll are sharing what you're doing right/finding interesting.

Since ya'll might appreciate this, I think there's an ultimate form of all these orchestrators out there that boils everything down to the "operator pattern" -- I call it "buhzaar" but I tried to get my thoughts out of the notebook a while ago[0]. It's almost like a completely normalized DB might be -- to strip an orchestrator down to it's bare minimum, which facilitates other processes that do resource provisioning and management. Then let people bring their own things that provision resources (and maybe you some "officially supported" ones but they all live separately and iterate separately).

I didn't quite put down all the thoughts I had but you think this is too much normalization (in the same way no one wants to do 7 joins)? You could argue that both nomad and k8s are denormalized (they intrinsically "know" how to provision/manage certain things) to a certain extent, and nomad just "bundles" less.

[0]: https://gitlab.com/buhzaar/design

Please feel free too, would love to chat about this. I think we think extremely similarly -- What you're trying to build is almost exactly what I'm trying to build, except I plan on getting my leverage from k8s (and eventually my own thing that I'm working on called buhzaar which aims to be simpler than k8s).

Regarding turning Dockerfiles into a MicroVM: https://gruchalski.com/posts/2021-03-23-introducing-firebuil..., on GitHub: https://github.com/combust-labs/firebuild. This could get you started.

Disclaimer: I’m the author.

> See, and in almost all of my use-cases, I really do. I do HPC computing, which is almost always a multi-tenant environment.

Maybe you need firecracker with something along the lines of https://github.com/combust-labs/firebuild?

I was the one who posted that question and I deleted it because I understood I was asking to compare apple to oranges. Firecracker uses KVM under the hood. With regards to the overhead, sure, there's going to be resources needed but firecracker is pretty good at running VMs will low level overhead. We're talking thousands on a single host.

I've done some fair share of evaluating firecracker for https://github.com/combust-labs/firebuild and the need to provision is red herring. firebuild can run a VM directly from a Dockerfile and Docker image. Fly.io team does something similar. It's basically a fully functional app out of the container within a matter of milliseconds.

What would be nice is to see a direct comparison between your solution and firecracker.

I am working on something like this: https://github.com/combust-labs/firebuild.

From a Dockerfile, it's not as simple without creating an image first.