humility VS probe-rs

A lot of questions in there! Taking these in order:

1. We aren't making standalone servers: the Oxide compute sled comes in the Oxide rack. So are not (and do not intend to be) a drop in replacement for extant rack mounted servers.

2. We have taken a fundamentally different approach to firmware, with a true root of trust that can attest to the service processor -- which can turn attest to the system software. This prompts a lot of questions (e.g., who attests to the root of trust?), and there is a LOT to say about this; look for us to talk a lot more about this

3. In stark contrast (sadly) to nearly everyone else in the server space, the firmware we are developing is entirely open source. More details on that can be found in Cliff Biffle's 2021 OSFC talk and the Hubris and Humility repos.[0][1][2]

4. Definitely not vaporware! We are in the process of shipping to our first customers; you can follow our progress in our Oxide and Friends podcast.[3]

[0] https://www.osfc.io/2021/talks/on-hubris-and-humility-develo...

[1] https://github.com/oxidecomputer/hubris

[2] https://github.com/oxidecomputer/humility

[3] https://oxide-and-friends.transistor.fm/

It's a mix of embedded work and improving the system's tooling (faster builds, debugger support, etc)

Other folks have mentioned this, but it's important to understand the limitations of Rust with respect to safety. In particular: every stack operation is -- at some level -- an unsafe operation as it operates without a bounds check. This isn't Rust's fault per se; non-segmented architectures don't have an architecturally defined way to know the stack base. As a result, even an entirely safe Rust program can make an illegal access to memory that results in fatal program failure. That, of course, assumes memory protection; if you don't have memory protection (or, like many embedded operating systems, you don't make use of it), stack overflows will plow into adjacent memory.

But wait, it gets worse: stack overflows are often not due to infinite stack consumption (e.g., recursion) but rather simply going deep on an unusual code path. If stack consumption just goes slightly beyond the base of the stack and there is no memory protection, this is corrupt-and-run -- and you are left debugging a problem that looks every bit like a gnarly data race in an unsafe programming language. And this problem becomes especially acute when memory is scarce: you really don't want a tiny embedded system to be dedicating a bunch of its memory to stack space that will never ("never") be used, so you make the stacks as tight as possible -- making stack overflows in fact much more likely.

Indeed, even with the MPU, these problems were acute in the development of Hubris: we originally put the stack at the top of a task's data space, and its data at the bottom -- and we found that tasks that only slightly exceeded their stack (rather than running all of the way through its data and into the protection boundary) were corrupting themselves with difficult-to-debug failures. We flipped the order to assure that every stack overflow hit the protection boundary[0], which required us to be much more intentional about the stack versus data split -- but had the added benefit of allowing us to add debugging support for it.[1]

Stack overflows are still pesky (and still a leading cause of task death!), but without the MPU, each one of these stack overflows would be data corruption -- answering for us viscerally what we "need the MPU for."

[0] https://github.com/oxidecomputer/hubris/commit/d75e832931f67...

[1] https://github.com/oxidecomputer/humility#humility-stackmarg...

In addition to Cliff's talk/blog -- which are absolutely outstanding -- I would recommend listening to the Twitter Space we did on Hubris and Humility last week.[0] It was a really fun conversation, and it also serves as a bit of a B-side for the talk in that it goes into some of the subtler details that we feel are important, but didn't quite rise to the level of the presentation. And of course, be sure to check out the source itself![1][2]

[0] https://www.youtube.com/watch?v=cypmufnPfLw

[1] https://github.com/oxidecomputer/hubris

[2] https://github.com/oxidecomputer/humility

Humility (the debugger)

Want debugging capabilities with anything with an am st link/jtag/other compatible probe? https://probe.rs/

/uj The SVG usage looks to be in a visualisation helper. Maybe ought to be a separate module, but I've seen worse.

There is a command to list supported chips for *flashing* in all our CLI tools. Furthermore https://github.com/probe-rs/probe-rs/tree/master/probe-rs/targets contains all targets for which we support *flashing*.

Under the hood, cargo-embed, knurling's probe-run, and Humility are all built atop probe-rs (https://probe.rs/) to provide debugging - I think in this case it's actually a great example of cooperation between projects! Probe-rs has received PRs from both Knurling and Oxide devs; it provides the common interface to use various types of debug hardware and talk to various types of microcontroller cores, essentially replacing OpenOCD.

Ryzen 3700X, 3200Mhz DDR4 tower builds http://probe.rs in 1m47s with fans fully spinning. M1 Pro: zero noise, 1m21s. Ryzen 3950X is on par with the M1 Pro. Actual speed difference will very much depend on your actual workload. Compiling for aarch64 apparently is more efficient than for amd64. Which is why compiling for you locally gives much more speed benefit than some generic benchmarks would indicate :) Also, you cannot forget that I can easily work for 10hrs on my MBP M1 Pro with rust-analyzer and frequent compiles running. You can forget that with any other suggested "on par" notebook. They will drain your battery instantly. Also, with the same thermal mass, other laptop builds will go into throttling much faster, which will lead to slower effective speed. XPS laptops and old macbooks know this issue very well :)

Anyone who fancies doing some coding for fun should have a go at an embedded project using rust's tools. Not just embassy, but the PACs, the HALs, probe-rs (probe-rs is bonkers good), and the community on matrix. My bet is on rust embedded seeing huge growth in the next few years.

probe-rs (embedded debugging toolkit for ARM and RISC-V which is supposed to be used in place of the above if you code with it as it aims to fully replace the GDB portion of the stack for Rust embedded development)

cargo install --git https://github.com/probe-rs/probe-rs probe-rs-debugger