hcloud-cloud-controller-manager VS cluster-api-provider-hetzner

export KUBECONFIG=hetzner-cluster-kubeconfig.yaml # Install Hetzner CCM kubectl apply -f https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm.yaml # Install Flannel CNI - You can use your preferred CNI instead, e.g. Cilium kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

When it comes to load balancing, I think the hcloud-cloud-controller-manager[1] is probably your best bet, and although I haven't tested it, I'm sure it can be coerced into some kind of working configuration with the vSwitch/Cloud Network coupling, even none of cluster is actually Cloud-based.

I haven't used Sidero Omni yet, but if it's as well architected as Talos is, I'm sure it's an excellent solution. It still leaves open the question of ordering and provisioning the servers themselves. For simpler use-cases it wouldn't be too difficult to hack together a script to interact with the Hetzner Robot API to achieve this goal, but if I wanted any level of robustness, and if you'll excuse the shameless plug, I think I'd write a custom operator in Rust using my hrobot-rs[2] library :)

[1] https://github.com/hetznercloud/hcloud-cloud-controller-mana...

Did you actually use the Hetzner LBs or did you just send traffic to the NodePort?

It's not even close to major public cloud providers, but this is my setup:

* https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne... (Terraform, Kubernetes bootstrap)

* Flux for CI

* nginx-ingress + Hetzner Loadbalancer (thanks to https://github.com/hetznercloud/hcloud-cloud-controller-mana...)

* Hetzner storage volumes (thanks to https://github.com/hetznercloud/csi-driver)

Kube-Hetzner supports Hetzner Cloud loadbalancers and volumes out of the box, though it also supports other components.

I plan to do the same or have already implemented it. I use the Hetzner private networks with the hcloud-cloud-controller-manager https://github.com/hetznercloud/hcloud-cloud-controller-manager. This creates routes so that the nodes are also connected to each other in the private network. All kubernetes resources listen on the private network. Except for the Kubernetes API, which listens on the external IP of the master so I can use kubectl from home. The actual gateway into the cluster is controlled by a load balancer from Hetzner.

I want to use Hetzner's Load Balancer as a Nginx Ingress controller. I follow this suggestion to install Nginx Ingress via helm.

Ive also been using Cluster-API + Cluster-API-Provider-Hetzner

https://github.com/syself/cluster-api-provider-hetzner

works rock solid

The Cluster API Provider Hetzner (CAPH) is an open-source project (maintained by Syself and the community; not a Hetzner project) that allows you to leverage the capabilities of Cluster API to manage highly-available Kubernetes clusters on both Hetzner baremetal servers (Robot) and Hetzner cloud instances.

Hetzner Cloud is officially supported, but that means setting up VPSs in Hetzner's Cloud offering, whereas this project was intended as a more or less independent pure bare-metal cluster. I see they offer Bare Metal support as well, but I haven't dived too deep into it.

I haven't used KubeOne, but I have previously used Syself's https://github.com/syself/cluster-api-provider-hetzner which I believe works in a similar fashion. I think the approach is very interesting and plays right into the Kubernetes Operator playbook and its self-healing ambitions.

That being said, the complexity of the approach, probably in trying to span and resolve inconsistencies across such a wide landscape of providers, caused me quite a bit of grief. I eventually abandoned this approach after having some operator somewhere consistently attempt and fail to spin up a secondary control plane VPS against my wishes. After poring over loads of documentation and half a dozen CRDs in an attempt to resolve it, I threw in my hat.

Of course, Kubermatic is not Syself, and this was about a year ago, so it is entirely possible that both projects are absolutely superb solutions to the problem at this point.

For anyone interested in Kubernetes on Hetzner, there's a really interesting CAPI provider being actively developed:

https://github.com/syself/cluster-api-provider-hetzner

Just a hint running ML Workloads on Hetzner is pretty cheap! You could use for managing k8s: https://github.com/syself/cluster-api-provider-hetzner

we (Syself) release Cluster-API Provider Hetzner v1.0.0-beta.16.

ah okay they come from the upstream cluster-api project. The caph project implements only the infrastructure provider part of Cluster API.

you can easily use it with https://github.com/syself/cluster-api-provider-hetzner