Tokthr Alternatives

tokthr reviews and mentions

• StoneKnifeForth
  5 projects | news.ycombinator.com | 17 Jul 2022

  I'm glad you enjoyed it! Have you tried running it?

  One could reasonably argue that SKF isn't really "booting" on an old 386 PC; though it'll probably run on one, you need to get Linux running on the PC first, because SKF can't read the input source code from disk or write the output executable to disk on its own, and it needs something to load it into memory and start it running as well. And, unlike real Forths, it's a batch-mode compiler: it can't be used interactively. The sense in which it's a "tiny bootstrap" is that it's a compiler that compiles itself, not the sense of "bootstrapping" that means to load oneself into RAM.

  — ⁂ —

  In 02007 I wrote https://github.com/kragen/tokthr as an investigation into precisely the question of "just how small a fully functional Forth environment could be packed". By using bytecode ("token threading") rather than DTC or ITC, tokthr provides "90% of" a traditional interactive Forth environment in about 1400 bytes, all bytecode except for a 239-byte machine-code core. Unfortunately, because it's incomplete, it's impossible to know whether the remaining 10% of the functionality requires adding another 10% to the code or another 90%—as in the old joke about how the first 90% of the project takes the first 90% of the schedule, and the remaining 10% takes the other 90%.

  My best guess, though, is that it'd be about 1700 bytes, so in 2KiB you'd have about 350 bytes left for the user program, which is probably about 64–128 lines of code. Interpretation can be slow, but it sure makes for denser code, and you don't need to have two copies of the interpreted code in RAM when you're bootstrapping.

  There have definitely been 8KiB interactive Forth systems, and I think there have been 4KiB ones, which was also the size Wozniak required for a BASIC interpreter on the Apple. tokthr suggests that 2KiB might be achievable for a stripped-down Forth.

  Óscar Toledo G.'s bootOS https://github.com/nanochess/bootOS is a particularly interesting non-Forth system in this OS/IDE/debugger vein, because it has enough functionality to write programs (in hexadecimal machine code), load them from disk, edit them, and save them, all in 512 bytes of code. It does require somewhat more than 512 bytes of RAM, but still very little—much less than has ever been present on a machine that it could run on.

  — ⁂ —

  My thinking on Forth, which is something I'm an amateur at, has changed a little bit since I wrote tokthr and SKF. I think that if you evaluate Forth as a programming language, it's generally going to look pretty deficient. It's more bug-prone and harder to read than alternatives like C, Lisp, BASIC, or assembly language, although it does support compile-time metaprogramming and powerful abstraction facilities.

  But if you ask the question, "What's the simplest usable development environment I can build?" the answer starts to look a lot like Forth, especially if you're running on a poorly characterized piece of hardware where you need to prioritize interactive experimentation with I/O ports. It gives you an interactive, scriptable environment for poking around, similar to what Tcl gives you https://yosefk.com/blog/i-cant-believe-im-praising-tcl.html but in 4 kilobytes rather than the megabyte and a half Tcl demands. And the environment includes not just a scriptable command line and the usual programming-language things like arithmetic, subroutines, variables, and arrays, but also compile-time metaprogramming, virtual memory, an editor that allows you to recompile code at screenful granularity (usually), a memory dumper, access to raw hardware devices, an assembler (usually), sometimes multithreading, and a sort of debugger.

  As in BASIC or Lisp or Python or Tcl, your UI is a REPL, so your programming language is also your command language; but, unlike in BASIC, your command language is extensible, so in many cases it can also work as the user interface to your application.

  In BASIC or in a C debugger you number the lines of code in your program; in Forth you name them, so you can "single-step" your program by running them one at a time. Then the ? word gives you a quick way to examine your variable values interactively, again as in a debugger. Like, here's me trying to interactively debug a square-root routine into existence in GForth on Linux:

      variable square variable guess  ok

• StoneKnifeForth (With a Metacircular Compiler)
  6 projects | news.ycombinator.com | 18 Feb 2021

  SKF's predecessor tokthr was derived from Jonesforth: https://github.com/kragen/tokthr/blob/master/tokthr.S

  IIRC the reason I only allowed single-byte identifiers was to simplify the symbol table, which is implemented by these four lines of code:

      : Type Four* header 6144 + + ; ( Table of definition Types: 1=code, 2=data )