humility VS tock

A lot of questions in there! Taking these in order:

1. We aren't making standalone servers: the Oxide compute sled comes in the Oxide rack. So are not (and do not intend to be) a drop in replacement for extant rack mounted servers.

2. We have taken a fundamentally different approach to firmware, with a true root of trust that can attest to the service processor -- which can turn attest to the system software. This prompts a lot of questions (e.g., who attests to the root of trust?), and there is a LOT to say about this; look for us to talk a lot more about this

3. In stark contrast (sadly) to nearly everyone else in the server space, the firmware we are developing is entirely open source. More details on that can be found in Cliff Biffle's 2021 OSFC talk and the Hubris and Humility repos.[0][1][2]

4. Definitely not vaporware! We are in the process of shipping to our first customers; you can follow our progress in our Oxide and Friends podcast.[3]

[0] https://www.osfc.io/2021/talks/on-hubris-and-humility-develo...

[1] https://github.com/oxidecomputer/hubris

[2] https://github.com/oxidecomputer/humility

[3] https://oxide-and-friends.transistor.fm/

It's a mix of embedded work and improving the system's tooling (faster builds, debugger support, etc)

Other folks have mentioned this, but it's important to understand the limitations of Rust with respect to safety. In particular: every stack operation is -- at some level -- an unsafe operation as it operates without a bounds check. This isn't Rust's fault per se; non-segmented architectures don't have an architecturally defined way to know the stack base. As a result, even an entirely safe Rust program can make an illegal access to memory that results in fatal program failure. That, of course, assumes memory protection; if you don't have memory protection (or, like many embedded operating systems, you don't make use of it), stack overflows will plow into adjacent memory.

But wait, it gets worse: stack overflows are often not due to infinite stack consumption (e.g., recursion) but rather simply going deep on an unusual code path. If stack consumption just goes slightly beyond the base of the stack and there is no memory protection, this is corrupt-and-run -- and you are left debugging a problem that looks every bit like a gnarly data race in an unsafe programming language. And this problem becomes especially acute when memory is scarce: you really don't want a tiny embedded system to be dedicating a bunch of its memory to stack space that will never ("never") be used, so you make the stacks as tight as possible -- making stack overflows in fact much more likely.

Indeed, even with the MPU, these problems were acute in the development of Hubris: we originally put the stack at the top of a task's data space, and its data at the bottom -- and we found that tasks that only slightly exceeded their stack (rather than running all of the way through its data and into the protection boundary) were corrupting themselves with difficult-to-debug failures. We flipped the order to assure that every stack overflow hit the protection boundary[0], which required us to be much more intentional about the stack versus data split -- but had the added benefit of allowing us to add debugging support for it.[1]

Stack overflows are still pesky (and still a leading cause of task death!), but without the MPU, each one of these stack overflows would be data corruption -- answering for us viscerally what we "need the MPU for."

[0] https://github.com/oxidecomputer/hubris/commit/d75e832931f67...

[1] https://github.com/oxidecomputer/humility#humility-stackmarg...

In addition to Cliff's talk/blog -- which are absolutely outstanding -- I would recommend listening to the Twitter Space we did on Hubris and Humility last week.[0] It was a really fun conversation, and it also serves as a bit of a B-side for the talk in that it goes into some of the subtler details that we feel are important, but didn't quite rise to the level of the presentation. And of course, be sure to check out the source itself![1][2]

[0] https://www.youtube.com/watch?v=cypmufnPfLw

[1] https://github.com/oxidecomputer/hubris

[2] https://github.com/oxidecomputer/humility

Humility (the debugger)

Hi! This is Daniel from OxidOS Automotive (stating this for disclaimer purposes).

Yes, our OS is based on TockOS, and our CEO (Alex Radovici) is #7 in the contributors list (https://github.com/tock/tock/graphs/contributors), with other colleagues contributing in the past years.

I'm definitely not the best person to answer this, but honestly it's not bad. Here's an example of a moderately complex peripheral, the cortex-m MPU, and how one rust OS handles it:

https://github.com/tock/tock/blob/3a0527d586702b8ae8cb242391...

Reads and writes turn into volatile reads, so everything works out under the hood. You get the benefits of everything having good names, declared sizes, and proper typing on your register accesses. You can extend that to bit accesses as well.

Rust still has a few areas it isn't competitive in, like your hyper limited or obscure chips (e.g. 8051s, XAP), mature tooling around formal methods, and a certification story for safety critical code. People are working on these latter two issues (e.g. ferrocene) and supposedly very close to public delivery, but you know how slow the industry is to adopt new things even then.

There's Tock (https://www.tockos.org/), which is written in Rust (with sprinkles of assembly).

Yeah, and I like I mentioned in the earlier comment, omitting the frame pointer reduces code size by 10% on RISC-V targets, which is huge when dealing with embedded flash: https://github.com/tock/tock/pull/1660

Tock is an excellent embedded OS written in Rust and has some good industrial support. I think Tock gets a lot of stuff right and I highly recommend some of the talks the developers gave on it.

Unfortunately, it increases the code size by 10%. I was looking into this just last week, and can confirm that it's still a problem on the latest version of Rust nightly: https://github.com/tock/tock/pull/1660

I wish we could have frame pointers, because they would make working in embedded land so much easier and more reliable, but a 10% increase in code size just isn't worth it.

TockOS was the first rust RTOS I found. Coincidentally, it has had support for the esp32c3 for over a year now.