bootOS VS bootstrap-seeds

wow, this is impressive.

I wrote a similar x86-16 assembler in < 512 B of x86-16 assembly, and this seems much more difficult <https://github.com/kvakil/0asm/>. I did find a lot of similar tricks were helpful: using gadgets and hashes. Once trick I don't see in sectorc which shaved quite a bit off of 0asm was self-modifying code, which 0asm uses to "change" to the second-pass of the assembler. (I wrote some other techniques here: <https://kvakil.me/posts/asmkoan.html>.)

bootOS (<https://github.com/nanochess/bootOS>) and other tools by the author are also amazing works of assembly golf.

I'm glad you enjoyed it! Have you tried running it?

One could reasonably argue that SKF isn't really "booting" on an old 386 PC; though it'll probably run on one, you need to get Linux running on the PC first, because SKF can't read the input source code from disk or write the output executable to disk on its own, and it needs something to load it into memory and start it running as well. And, unlike real Forths, it's a batch-mode compiler: it can't be used interactively. The sense in which it's a "tiny bootstrap" is that it's a compiler that compiles itself, not the sense of "bootstrapping" that means to load oneself into RAM.

— ⁂ —

In 02007 I wrote https://github.com/kragen/tokthr as an investigation into precisely the question of "just how small a fully functional Forth environment could be packed". By using bytecode ("token threading") rather than DTC or ITC, tokthr provides "90% of" a traditional interactive Forth environment in about 1400 bytes, all bytecode except for a 239-byte machine-code core. Unfortunately, because it's incomplete, it's impossible to know whether the remaining 10% of the functionality requires adding another 10% to the code or another 90%—as in the old joke about how the first 90% of the project takes the first 90% of the schedule, and the remaining 10% takes the other 90%.

My best guess, though, is that it'd be about 1700 bytes, so in 2KiB you'd have about 350 bytes left for the user program, which is probably about 64–128 lines of code. Interpretation can be slow, but it sure makes for denser code, and you don't need to have two copies of the interpreted code in RAM when you're bootstrapping.

There have definitely been 8KiB interactive Forth systems, and I think there have been 4KiB ones, which was also the size Wozniak required for a BASIC interpreter on the Apple. tokthr suggests that 2KiB might be achievable for a stripped-down Forth.

Óscar Toledo G.'s bootOS https://github.com/nanochess/bootOS is a particularly interesting non-Forth system in this OS/IDE/debugger vein, because it has enough functionality to write programs (in hexadecimal machine code), load them from disk, edit them, and save them, all in 512 bytes of code. It does require somewhat more than 512 bytes of RAM, but still very little—much less than has ever been present on a machine that it could run on.

— ⁂ —

My thinking on Forth, which is something I'm an amateur at, has changed a little bit since I wrote tokthr and SKF. I think that if you evaluate Forth as a programming language, it's generally going to look pretty deficient. It's more bug-prone and harder to read than alternatives like C, Lisp, BASIC, or assembly language, although it does support compile-time metaprogramming and powerful abstraction facilities.

But if you ask the question, "What's the simplest usable development environment I can build?" the answer starts to look a lot like Forth, especially if you're running on a poorly characterized piece of hardware where you need to prioritize interactive experimentation with I/O ports. It gives you an interactive, scriptable environment for poking around, similar to what Tcl gives you https://yosefk.com/blog/i-cant-believe-im-praising-tcl.html but in 4 kilobytes rather than the megabyte and a half Tcl demands. And the environment includes not just a scriptable command line and the usual programming-language things like arithmetic, subroutines, variables, and arrays, but also compile-time metaprogramming, virtual memory, an editor that allows you to recompile code at screenful granularity (usually), a memory dumper, access to raw hardware devices, an assembler (usually), sometimes multithreading, and a sort of debugger.

As in BASIC or Lisp or Python or Tcl, your UI is a REPL, so your programming language is also your command language; but, unlike in BASIC, your command language is extensible, so in many cases it can also work as the user interface to your application.

In BASIC or in a C debugger you number the lines of code in your program; in Forth you name them, so you can "single-step" your program by running them one at a time. Then the ? word gives you a quick way to examine your variable values interactively, again as in a debugger. Like, here's me trying to interactively debug a square-root routine into existence in GForth on Linux:

    variable square variable guess  ok

He also wrote bootOS, "a monolithic operating system in 512 bytes of x86 machine code."

https://github.com/nanochess/bootOS

This[0] is basically the hand-documentation of those bytes then. Handwritten ELF header and assembly code.

[0] https://github.com/oriansj/bootstrap-seeds/blob/master/POSIX...

The bootstrap seed, https://github.com/oriansj/bootstrap-seeds/blob/master/POSIX..., is a tiny interpreter that takes a much larger program written in a special-purpose, bytecode-based language. This proceeds in turn once or twice more--special purpose program generating another interpreter for another special-purpose language--until you end up with a minimal Scheme interpreter, which then can be used to execute a C compiler program.

All of this is incredible work, but a minimal C-subset compiler in under 512 bytes seems like a unique achievement.

There is also live-bootstrap which uses a similar bootstrap chain to Guix (stage0 -> Mes -> tcc -> gcc), but without needing Guile/guix-daemon binaries etc. The whole thing starts with just a 357-byte binary seed (source)!

Yeah, it's a binary blob, but it's small enough to be easily auditable. Anyone with some knowledge of x86 assembly can read the annotated version [1] and verify that it does what it claims (which is to convert ASCII hex with comments into binary).

You're right, it also requires a Linux kernel, and of course, you also have to trust the hardware you're running it on. Still, it reduces the amount of stuff we have to take for granted as trusted, which I think is a good thing. (I'm not involved in the project, just an admirer).

[1]: https://github.com/oriansj/bootstrap-seeds/blob/b09a8b8cbcb6...

stage0-posix just gained initial support for RISC-V (64-bit). It starts with 392 byte hex assembler, 361 byte "shell" and bootstraps simple linker (hex2), macro assembler (M0). Then it builds cc_riscv64 RISC-V compiler written in RISC-V assembly and uses it to build simple C compiler written in C (M2-Planet). Then it builds a few extra utilities (cp, mkdir, untar, ungz, sha256sum, chmod)

From a security point of view the only thing that gentoo users need to achieve similar levels of security is a bootstrapped compiler from a known good seed. The source code is already deterministic by definition. After that all you need is a compiler bootstrapped via something like https://github.com/oriansj/bootstrap-seeds which can be independently verified. It would probably be useful to be able to have independent bootstraps arrive at the same binary output for a compiler, but probably only as an option. Ultimately way less work for the same level of security.