RPi4 VS k3s

Correct. It does not as shipped. However, the use of this project will bring the firmware into system ready spec, so it can boot with a standard aarch64 UEFI image: https://github.com/pftf/RPi4

are you using the uefi firmware from https://github.com/pftf/RPi4 or are you trying to boot through the gpio serial header?I don't think the pi can boot on its own through uboot unless your using a serial/usb connection

Looks like you can use Grub on UEFI ARM systems, but Raspberry Pi isn't natively running UEFI. https://github.com/pftf/RPi4

The rpi4 has uefi firmware available, this allows you to boot any generic uefi aarch64 image, you no longer need rpi specific images.

https://github.com/pftf/RPi4

Straight out of the box, the image wouldn't boot, said that start.elf was invalid, so I went to https://github.com/pftf/RPi4/releases as suggested in the Readme.md file in the EFI partition. I installed that (version 1.34) over the existing EFI partition and tried again. That booted up the kernel, but it apparently died when it enabled the interrupt controller. The last messages are about armgic0.

Maybe similar discussion on github:

You can use these firmware images for UEFI as well as install with the arm ISO. I didn't have graphics acceleration that way, but it might be an easy fix.

I'm guessing these are not SystemReady certified with UEFI firmware and require "bespoke" preinstalled arm images?

https://www.arm.com/architecture/system-architectures/system...

https://developer.arm.com/documentation/102677/0100/UEFI-req...

I have three SystemReady arm devices and it's pretty awesome to be able to just boot an aarch64 live ISO and install. The experience is the same for running vms via ESXi arm edition.

Nvidia Jetson AGX Xavier - https://developer.nvidia.com/embedded/downloads#?search=uefi

Honeycomb LX2 - https://github.com/SolidRun/lx2160a_uefi

RPI4 - https://github.com/pftf/RPi4

It can be tedious building/provisioning the firmware but once complete they are ready for any aarch64 uefi iso.

What is annoying however is when distros don't ship an aarch64 uefi iso - but instead choose to build a zillion device specific "preinstalled" arm images. (looking at you manjaro)

The list of supported devices for ESXi arm edition is a great place to start for identifying options and is constantly updated.

https://flings.vmware.com/esxi-arm-edition

  Raspberry-Pi-4

The only reason why I am was trying to build the image was because I wanted to move stuff as mainline as possible and was worried that any installation made with the help of RPi4 UEFI firmware would stop booting after a while.

Most ARM hardware is cellphones, raspberry pi and the Mac M1, which certainly aren't that type.

But a lot of ARM hardware is that type. The keywords are SBSA / SBBR / SystemReady. If your hardware is SBBR compatible then Fedora and Ubuntu's ARM64 iso, and Windows ARM64, downloaded from their website, will at least boot fine (drivers are a different question as always).

There's a good list of supported hardware in the lower half of https://community.arm.com/arm-community-blogs/b/architecture... . Many systems from Avantek, Gigabyte, NXP, Marvell, Solidrun etc are standardizing on this way of booting.

DeviceTree is low-level enough that you can implement UEFI on top of it. There's a UEFI port for the Raspberry Pi 4 at https://rpi4-uefi.dev/ that produces an SBBR layer, allowing it to boot any off-the-shelf ARM64 SBBR distro.

Opinionated meaning it picks, install, patches your CNI/Ingress/Load Balancer/DNS Server/Metrics Server/Monitoring Setup.

k3s is probably most well known as it ships with bunch of preinstall software: https://github.com/k3s-io/k3s so you can just start throwing yaml files at cluster and handling workloads. It's what I use for my homelab.

Paid things I've heard of include OpenStack and SideroLabs. Haven't used personally by SRE coworkers say good things about them.

For self-hosting I've found https://k3s.io to be really good from the SUSE people. Works on basically any Linux distro and makes self-hosting k8s not miserable.

Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319

For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.

We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.

K3S: is a lightweight distribution of Kubernetes that is designed for resource-constrained environments. It is an excellent option for running Kubernetes on a virtual machine or cloud server.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Kubernetes - k3s distribution

I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.

You should be familiar with Kubernetes and have set up a Kubernetes cluster. I recommend k3s.