axiom VS cligen

I'm creating a hobby operating system in Nim (early stages). After an initial attempt[0] I decided to start over[1] and document my journey in building it[2]. The focus for me is on learning low-level systems programming and enjoying the journey, rather than building something production ready.

[0] https://github.com/khaledh/axiom

[1] https://github.com/khaledh/fusion

[2] https://0xc0ffee.netlify.app/osdev/01-intro.html

> kernel developers do not allow third party runtimes in the kernel. Even meager Rust's "panic" runtime is a contentious

Much in Linux is contentious :-) which is why the module system is nice. A kernel module for C code requires no permission from Linux-core unless you need it distributed with the kernel (which, yes, might be required for "credibility" - but critically also might not). It may require many decls to access various kernel APIs, but those can be (semi-)automated or just done as-needed. So, Linux kernel policy is not so relevant (at best) which is what I meant by "no special support" (admittedly brief). Kernel coding is always a bit trickier, and you may need to build up some support code to make integration nice, though as well as decl generators.

> Can one disable runtime in Nim completely -- no GC, no exceptions?

To answer your question, and as discussed elsewhere in this subthread, Nim has many options for memory management.. only stdlib seq/string really needs automatic methods. One can disable the runtime completely via os:standalone and statically check that no exceptions are raised with Nim's effect system (and there are also both setjmp & goto based exception impls which may/may not be workable in Linux/BSD kernel module settings). As "proof more by example", a few people have written OS kernels in Nim recently[1,2] and there was another toy kernel long ago[3].

People have also written OS kernels in Go which "has a GC and runtime".[4] So, I acknowledge it's not quite the same example, but I also see no fundamental blockers for kernel modules.

[1] https://github.com/khaledh/axiom

[2] https://prosepoetrycode.potterpcs.net/2023/01/a-barebones-ke...

[3] https://github.com/dom96/nimkernel

[4] https://github.com/mit-pdos/biscuit/

I gave Rust a few chances, and always came out hating its complexity. I needed a systems programming language to develop a hobby OS[1], and Nim hit the sweet spot of being very ergonomic, optional GC, and great interop with C. I can drop down to assembly any time I want, or write a piece of C code to do something exotic, but the rest of the system is pure Nim. It's also quite fast.

[1] https://github.com/khaledh/axiom

I've used both to work on a hobby OS project (Nim[1], Zig[2]). I very much prefer Nim. Code is succinct, elegant, and lets you focus on your core logic rather than fighting the language.

Zig is nice and I like its optionals support and error handling approach. But I was put off by its noisy syntax, e.g. !?[]u8 to represent an error union of an optional pointer to a many-pointer of uint8. Also having to prepare and weave allocators throughout most of the code that needs to dynamically allocate (which is most of the code) gets in the way of the main logic. Even little things like string concatenation or formatting becomes a chore. Zig also doesn't have dynamic dispatch, which makes polymorphic code hard to write; you have to work around it through some form of duck typing. In the end I realized that Zig is not for me.

[1] https://github.com/khaledh/axiom

Niceness is subjective, but Nim is just as valid an addition to that group. Nim compiles to C and has had an --os=standalone mode for like 10 years from its git history, and as mentioned else-thread (https://news.ycombinator.com/item?id=36506087) can be used for Linux kernel modules. Multiple people have written "stub OSes" in it (https://github.com/dom96/nimkernel & further along https://github.com/khaledh/axiom).

While it can use clang as a backend, Nim does not rely upon LLVM support like Zig or Rust (pre-gcc-rust working). Use on embedded devices is fairly popular: https://forum.nim-lang.org/search?q=embedded (or web search).

Latency-wise, for a time, video game programming was a perceived "adoption niche" or maybe "hook" for Nim and games often have stringent frame rendering deadlines. If you are interested in video games, you might appreciate https://github.com/shish/rosettaboy which covers all but Ada in your list with Nim being fastest (on one CPU/version/compiler/etc). Note, however, that cross-PL comparisons are often done by those with much "porting energy" but limited familiarity with any but a few of the PLs. A better way to view it is that "Nim responds well to optimization effort" (like C/Ada/C++/Rust/Zig).

If anyone is interested, I have a couple of implementations of booting under UEFI and getting a bunch of info about the system (don't expect a functioning system, they just boot and dump some info):

Nim: https://github.com/khaledh/axiom

Zig: https://github.com/khaledh/axiom-zig (this one goes into depth in disassembling ACPI DSDT bytecode)

I wasn't ready to share it yet, but here it goes[1]. It's at a very early stage, but should give you an idea of how to get things up and running under Nim.

I didn't avoid malloc. I provided a simple bump pointer based heap to get things going. Later I'll have to separate things into a UEFI bootloader and a proper kernel image, each with its own allocator (the bootloader will use UEFI memory allocation services, and the kernel will have its own heap).

[1] https://github.com/khaledh/axiom

There is also generating the whole thing from a function signature (e.g. https://github.com/c-blake/cligen ) since then CLauthors need not learn a new spec language, but then CLauthors must add back in helpful usage metadata/semantics and still need to learn a library API (but I like how those two things can be "gradual"). It's a hard space in which to find perfection, but I wish you luck in your attempt!

cligen also allows End-CL-users to adjust colorization of --help output like https://github.com/c-blake/cligen/blob/master/screenshots/di... using something like https://github.com/c-blake/cligen/wiki/Dark-BG-Config-File

Last I knew, the argparse backing most Py CLI solutions did not support such easier (for many) to read help text, but the PyUniverse is too vast to be sure without much related work searching.

20 milliseconds? On my 7 year old Linux box, this little Nim program https://github.com/c-blake/bu/blob/main/wsz.nim runs to completion in 275 microseconds when fully statically linked with musl libc on Linux. That's with a stripped environment (with `env -i`). It takes more like 318 microseconds with my usual 54 environment variables. The program only does about 17 system calls, though.

Additionally, https://github.com/c-blake/cligen makes decent CLI tools a real breeze. If you like some of Go's qualities but the language seems too limited, you might like Nim: https://nim-lang.org. I generally find getting good performance much less of a challenge with Nim, but Nim is undeniably less well known with a smaller ecosystem and less corporate backing.

If you find this article interesting and are curious about Nim then you would probably also be curious about https://github.com/c-blake/cligen

That allows adding just 1-line to a module to add a pretty complete CLI and then a string per parameter to properly document options (assuming an existing API using keyword arguments).

It's also not hard to compile & link a static ELF binary with Nim.. I do it with MUSL libc on Linux all the time. I just toss into my ~/.config/nim/nim.cfg:

    @if musl:  # make nim c -d:musl .. foo static-link `foo` with musl

Sure. No problem.

Even Windows has popen these days. There are some tiny popenr/popenw wrappers in https://github.com/c-blake/cligen/blob/master/cligen/osUt.ni...

Depending upon how balanced work is on either side of the pipe, you usually can even get parallel speed-up on multicore with almost no work. For example, there is no need to use quote-escaped CSV parsing libraries when you just read from a popen()d translator program producing an easier format: https://github.com/c-blake/nio/blob/main/utils/c2tsv.nim

Nim is terse yet general and can be made even more so with effort. E.g., You can gin up a little framework that is even more terse than awk yet statically typed and trivially convertible to run much faster like https://github.com/c-blake/bu/blob/main/doc/rp.md

You can statically introspect code to then generate related/translated ASTs to create nearly frictionless helper facilities like https://github.com/c-blake/cligen .

You can do all of this without any real run-time speed sacrifices, depending upon the level of effort you put in / your expertise. Since it generates C/C++ or Javascript you get all the abilities of backend compilers almost out of the box, like profile-guided-optimization or for JS JIT compilation.

It's more that those tools were what come to mind when I specifically think of my exposure to the existence of rust. Its perhaps not that the tools were there, but that they were well known (and known for being written in rust).

Anecdatapoint - I've never heard of literally a single one of the utilities listed on the bu page.

Regarding cligen, right from the start clap wins on producing idiomatic output. Compare: https://github.com/c-blake/cligen#cligen-a-native-api-inferr...

    Usage:

cool example would be this which is a CLI generation library. It lets you describe command line interfacs simply using function signatures

>Does nim have anything as polished and performant as clap and serde?

"Polished" and "high quality" are more subjective/implicitly about adoption, IMO. "Performant" has many dimensions. I just tested the Nim https://github.com/c-blake/cligen vs clap: cligen used 5X less object file space (with all size optimization tweaks enabled in both), 20% less run-time memory for large argument lists, and the same run-time per argument (with march=native equivalents on both, within statistical noise). cligen has many features - "did you mean?/suggestions", color generated help and all that - I do not see obvious feature in clap docs missing in cligen. The Nim binary serde showing is unlikely as good but there are like 10 JSON packages and that seems maybe your primary concern.

More to add color your point than disagree (and follow up on my "adoption") - your ideas about polish, quality, docs, etc. are part of feedback loop(s) you mentioned. More users => Users complain (What is confusing? What is missing? etc.) => things get fixed/cleaned up/improved => More users. Besides "performant" being multi-dimensional, the feedback loop is more of a "cyclic graph". :-) While I probably prefer Nim as much or more as @netbioserror, I am not too shocked by the mindshare capture. It seems to happen every 5..10 years or so in prog.langs.

While many of your points are not invalid, tech is also a highly hype-driven & fad-driven realm. In my experience, the more experience with this meta-feature that someone has, the more skeptical they are of the latest thing (more rounds of regret, etc.). Also, that feedback graph is not a pure good. Things can get too popular too quickly with near permanent consequences. ipv4 got popular so quickly that we are still mostly stuck on it 40 years later as ipv6 struggles for penetration. Whatever your favorite PL is, it may also grow features too fast.

You are welcome. Thanks are too rarely offered. :-)

You may also be interested in word stemming ( such as used by snowball stemmer in https://github.com/c-blake/nimsearch ) or other NLP techniques, but I don't know how internationalized/multi-lingual that stuff is, but conceptually you might want "series of stemmed words" to be the content fragments of interest.

Similarity scores have many applications. Weights on graph of cancelled downloads ranked by size might be one. :)

Of course, for your specific "truncation" problem, you might also be able to just do an edit distance against the much smaller filenames and compare data prefixes in files or use a SHA256 of a content-based first slice. ( There are edit distance algos in Nim in https://github.com/c-blake/cligen/blob/master/cligen/textUt.... as well as in https://github.com/c-blake/suggest ).

Or, you could do a little program like ndup/sh/ndup to create a "mirrored file tree" of such content-based slices then you could use any true duplicate-file finder (like https://github.com/c-blake/bu/blob/main/dups.nim) on the little signature system to identify duplicates and go from path suffixes in those clusters back to the main filesystem. Of course, a single KV store within one or two files would be more efficient than thousands of tiny files. There are many possibilities.