RPi4 VS k3s

Correct. It does not as shipped. However, the use of this project will bring the firmware into system ready spec, so it can boot with a standard aarch64 UEFI image: https://github.com/pftf/RPi4

are you using the uefi firmware from https://github.com/pftf/RPi4 or are you trying to boot through the gpio serial header?I don't think the pi can boot on its own through uboot unless your using a serial/usb connection

Looks like you can use Grub on UEFI ARM systems, but Raspberry Pi isn't natively running UEFI. https://github.com/pftf/RPi4

The rpi4 has uefi firmware available, this allows you to boot any generic uefi aarch64 image, you no longer need rpi specific images.

https://github.com/pftf/RPi4

Straight out of the box, the image wouldn't boot, said that start.elf was invalid, so I went to https://github.com/pftf/RPi4/releases as suggested in the Readme.md file in the EFI partition. I installed that (version 1.34) over the existing EFI partition and tried again. That booted up the kernel, but it apparently died when it enabled the interrupt controller. The last messages are about armgic0.

Maybe similar discussion on github:

You can use these firmware images for UEFI as well as install with the arm ISO. I didn't have graphics acceleration that way, but it might be an easy fix.

I'm guessing these are not SystemReady certified with UEFI firmware and require "bespoke" preinstalled arm images?

https://www.arm.com/architecture/system-architectures/system...

https://developer.arm.com/documentation/102677/0100/UEFI-req...

I have three SystemReady arm devices and it's pretty awesome to be able to just boot an aarch64 live ISO and install. The experience is the same for running vms via ESXi arm edition.

Nvidia Jetson AGX Xavier - https://developer.nvidia.com/embedded/downloads#?search=uefi

Honeycomb LX2 - https://github.com/SolidRun/lx2160a_uefi

RPI4 - https://github.com/pftf/RPi4

It can be tedious building/provisioning the firmware but once complete they are ready for any aarch64 uefi iso.

What is annoying however is when distros don't ship an aarch64 uefi iso - but instead choose to build a zillion device specific "preinstalled" arm images. (looking at you manjaro)

The list of supported devices for ESXi arm edition is a great place to start for identifying options and is constantly updated.

https://flings.vmware.com/esxi-arm-edition

  Raspberry-Pi-4

The only reason why I am was trying to build the image was because I wanted to move stuff as mainline as possible and was worried that any installation made with the help of RPi4 UEFI firmware would stop booting after a while.

Most ARM hardware is cellphones, raspberry pi and the Mac M1, which certainly aren't that type.

But a lot of ARM hardware is that type. The keywords are SBSA / SBBR / SystemReady. If your hardware is SBBR compatible then Fedora and Ubuntu's ARM64 iso, and Windows ARM64, downloaded from their website, will at least boot fine (drivers are a different question as always).

There's a good list of supported hardware in the lower half of https://community.arm.com/arm-community-blogs/b/architecture... . Many systems from Avantek, Gigabyte, NXP, Marvell, Solidrun etc are standardizing on this way of booting.

DeviceTree is low-level enough that you can implement UEFI on top of it. There's a UEFI port for the Raspberry Pi 4 at https://rpi4-uefi.dev/ that produces an SBBR layer, allowing it to boot any off-the-shelf ARM64 SBBR distro.

Finally the main reason for us is the choice of runtime. We have very decent container runtimes (RKE, RHOS, K3s) available to deploy applications. We are very familiar with them, and they offer us a lot of functionality. These all support containers primarily.

I used K3S in my cluster because it is a lightweight, stripped-down version of Kubernetes that’s ideal for running on resource-constrained devices like Raspberry PIs I plan to use in my home lab cluster. K3S can be installed through a shell script:

In the last few years, we’ve witnessed the introduction of several new lightweight Kubernetes distributions. SUSE’s Rancher Labs k3s project was one of the earliest. Canonical now includes Microk8s in Ubuntu. And k0s is a single-binary Kubernetes distribution.

K3S is a Kubernetes distribution made by Rancher, made to be as lightweight as possible while being compatible with Kubernetes production standards.

Jeff led me to K3s using Ansible, a lightweight Kubernetes distribution that is perfect for my home cluster and a pre-defined way of installing it because I don't have pre-requirements nor the idea on how to set it up otherwise.

Still requires SRE but alot easier than it used to be: https://www.talos.dev/v1.8/

Also, k3s. https://k3s.io/

I'm torn between https://k0sproject.io and https://k3s.io to use in CI and production.

Any suggestions or personal experience?

What you also can do is starting with just a single node, incredibly easy to install with e.g. https://k3s.io/. You still have to invest the upfront effort to understand how it works but you can already reap a lot of benefits with a lot less complexity.

Kubernetes does not force you into the distributed systems hell, you can go that route later, or never.

> Gosh, I'm so happy I was able to jump of the k8s hype train. This is not something SMBs should be using. Now I happily manage my fleet of services without large infra overhead via my own paas over Docker Swarm. :)

I mean, I also use Docker Swarm and it's pretty good, especially with Portainer.

To me, the logical order of tools goes with scale a bit like this: Docker Compose --> Docker Swarm --> Hashicorp Nomad / Kubernetes

(with maybe Podman variety of tools where needed)

I've yet to see a company that really needs the latter group of options, but maybe that's because I work in a country that's on the smaller side of things.

All that being said, however, both Nomad and some K8s distributions like K3s https://k3s.io/ can be a fairly okay experience nowadays. It's just that it's also easy to end up with more complexity than you need. I wonder if it's going to be the meme about going full circle and me eventually just using shared hosting with PHP or something again, though so far containers feel like the "right" choice for shipping things reasonably quickly, while being in control of how resources are distributed.

The author is using https://k3s.io/ not the full k8s, so it doesn't have to be extremely expensive.