liblinux VS wasm3

> libc isn't really getting in the way here.

For the standard set of system calls, the libc is pretty great. For Linux-specific features, it could take years for glibc to gain support. Perhaps it's gotten better since then, perhaps it still takes years. I don't know.

Years ago I read about the tale of the getrandom system call and the quest to get glibc to support it:

https://lwn.net/Articles/711013/

A kernel hacker wrote in an email:

> maybe the kernel developers should support a libinux.a library that would allow us to bypass glibc when they are being non-helpful

That made a lot of sense to me. I took that concept and kind of ran with it. Started a liblinux project, essentially a libc with nothing but the thinnest possible system call wrappers. Researched quite a bit about glibc's attitude towards Linux to justify it:

https://github.com/matheusmoreira/liblinux#why

Eventually I discovered Linux was already doing the same thing with their own nolibc.h file which they were already using in their own tools. It was a single file back then, by now it's become a sprawling directory full of code:

https://github.com/torvalds/linux/tree/master/tools/include/...

Even asked Greg Kroah-Hartman on reddit about it once:

https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...

Since the kernel was developing their own awesome headers, I decided to drop liblinux and start lone instead. :)

It gives you access to 100% of Linux's system calls. It eliminates a lot of global state. It gets rid of a lot of legacy libc crap.

Years ago I wrote a fairly referenced rationale in my liblinux project:

https://github.com/matheusmoreira/liblinux/blob/master/READM...

> Now, do I think it would make total sense for syscall wrappers and NSS to be split into their own libs (or dbus interfaces maybe) with stable ABIs to enable other libc's, absolutely!

I worked on this a few years ago: liblinux.

https://github.com/matheusmoreira/liblinux

I'm not developing it anymore though because I found out the Linux kernel itself has a superior nolibc library:

https://github.com/torvalds/linux/tree/master/tools/include/...

It used to be a single header but it looks like they've recently organized it into a proper project!

I wonder if it will become some kind of official kernel library at some point. I asked Greg Kroah-Hartman about this and he mentioned there was once a klibc:

https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...

> This is something the BSD's got absolutely right.

BSDs, every other operating system really, force us to use the bundled C libraries and the C ABI. I think Linux's approach is better. It has a language-agnostic system call binary interface: it's just a simple calling convention and the system call instruction.

The right place for system call support is the compiler. We should have system_call keywords that cause it to emit code in the aforementioned calling convention. With this single keyword, it's possible to do program literally anything on Linux. Wrappers for every specific system call should be part of every language's standard library with language-specific types and semantics.

I'm not using this stuff professionally, it's just my own home lab's virtual machines with little services implemented as freestanding C programs. Not doing anything fancy right now, much of it was just to see if I could do it.

I've seen other people commenting here on HN saying they're using the same approach so it's defenitely not my invention.

I published some of my work in the form of a liblinux that I use to make system calls:

https://github.com/matheusmoreira/liblinux

I'm not developing it anymore though because I found out the kernel itself has a nolibc library:

https://github.com/torvalds/linux/tree/master/tools/include/...

It used to be a single header but it looks like they've organized it into a proper project.

That's awesome. I didn't know about rwildcard until now. Is it part of GMSL? I searched for rwildcard on gmsl.sourceforge.io but didn't find it.

I think my function is needlessly complicated compared to rwildcard. Here's my code:

https://github.com/matheusmoreira/liblinux/blob/modular-buil...

https://github.com/matheusmoreira/liblinux/blob/modular-buil...

The file? and directory? functions were inspired by GMSL.

I wrote a general recursion function. It takes a function to apply to lists and a function to compute whether an element is a base case.

The recursive file system traversal function applies a directory globbing function to the list of paths and has file? as base case.

The find function filters out any items not matching a given predicate function. It was my intention to provide predicates like C_file? and header_file? but I stopped developing that project before that happened.

I think rwildcard is probably simpler and more efficient!

As long as this is happening, might as well try some of my favorites: https://github.com/wasm3/wasm3, https://github.com/WebAssembly/wabt, https://github.com/bytecodealliance/wasmtime

This means that newly created wasm blobs will stop being able to run in wasm3.

On a side note, I can't help feeling sorry for the people that advocate for C over C++ when I see commits like https://github.com/wasm3/wasm3/commit/121575febe8aa1b544fbcb...

It can, wasm3 is a wasm interpretor ported to a lot of bare metal microcontrollers: https://github.com/wasm3/wasm3

On other benchmarks I'm seeing numbers closer to 20% slower, e.g. https://github.com/wasm3/wasm3/blob/main/docs/Performance.md and https://github.com/second-state/wasm32-wasi-benchmark. It's numerical code, which is the best case scenario for a native binary. It's much closer on an average web app or server workload, e.g. https://krausest.github.io/js-framework-benchmark/current.html - you can find WASM frameworks that beat most JS frameworks on there, but that is not as impressive considering the state of the JS ecosystem. Overall, it's already under 50%, and there is still plenty of room for improvement.

Can miniwasm share memory with the host? wasm3 doesn't allow this[1] and requires you to allocate VM memory and pass it to the host, but that has several downsides (some buffers come from external sources so this requires a memcpy; the VM memory location isn't stable so you can't store a pointer to it on the host; etc.).

I'm really interested in a fast interpreter-only Wasm VM that can allow the host to share some of its memory with the VM.

[1]: https://github.com/wasm3/wasm3/issues/114

> This product family will never run Linux, as developers will need to develop a new firmware from scratch. This is obviously not a problem for the earbuds, but a big limitation for the player

Perhaps not Linux, but I suspect there would be a place here for a Unix-like platform that feels familiar. If for example we could get wide-adoption of something like the JVM or wasm3 [0] on these platforms, code could become quite portable, despite wildly different architectures.

For example, Apache's NuttX [1] (that I first learned from Lupyuen, a guy making great progress working with Pine64 products).

> Processing wise, this chip is well sufficient for TWS headphones, but very inadequate for an audio player. It will not drive a good screen, it nor run high-resolution flacs or (probably?) support a high-quality, high-bandwidth codec. In fact, a first generation iPod Nano (retailing for $149 in 2006) had 16MB RAM, so over 16 times what the PinePod would offer. In fact, even the features of any custom firmware are limited from so little memory

I wouldn't call it time just yet. Displays can be interacted with intelligently (to reduce pixel bandwidth) and ultra high quality audio codecs offer diminishing returns, especially when you don't have a DAC or headphones to make the most of them.

My advice to Pine64 would be this:

1. Consolidate your product lines. The Pinebook is just a slower Pinebook Pro, just go with the Pinebook Pro. The PineTab is just a Pinebook without the keyboard, again I would consolidate this with the Pinebook Pro and just make the keyboard detachable.

2. The SBCs should just go straight into the device, thus creating a clear upgrade path for future products. If you want a PineBook Pro running Quartz, just swap the boards (of course with daughter boards for USB expansion, display driver, power, etc).

3. Don't be afraid to kill off products. The Pinebook and PineTab have never seen a new release. The PinePhone appears to be taking a back seat to the PinePhone Pro. The PineCube is basically DoA due to the processing power struggling to process the camera image.

More generally, try to do fewer things, but do them well.

[0] https://github.com/wasm3/wasm3

[1] https://nuttx.apache.org/docs/latest/

[2] https://lupyuen.github.io/articles/sensor

I think SIMD was a distraction to our conversation, most code doesn't use it and in the future the length agnostic, flexible vectors; https://github.com/WebAssembly/flexible-vectors/blob/master/... are a better solution. They are a lot like RVV; https://github.com/riscv/riscv-v-spec, research around vector processing is why RISC-V exists in the first place!

I was trying to find the smallest Rust Wasm interpreters I could find, I should have read the source first, I only really use wasmtime, but this one looks very interesting, zero deps, zero unsafe.

16.5kloc of Rust https://github.com/rhysd/wain

The most complete wasm env for small devices is wasm3

20kloc of C https://github.com/wasm3/wasm3

I get what you are saying as to be so small that there isn't a place of bugs to hide.

> “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.” CAR Hoare

Even a 100 line program can't be guaranteed to be free of bugs. These programs need embedded tests to ensure that the layer below them is functioning as intended. They cannot and should not run open loop. Speaking of 300+ reimplementations, I am sure that RISC-V has already exceeded that. The smallest readable implementation is like 200 lines of code; https://github.com/BrunoLevy/learn-fpga/blob/master/FemtoRV/...

I don't think Wasm suffers from the base extension issue you bring up. It will get larger, but 1.0 has the right algebraic properties to be useful forever. Wasm does require an environment, for archival purposes that environment should be written in Wasm, with api for instantiating more envs passed into the first env. There are two solutions to the Wasm generating and calling Wasm problem. First would be a trampoline, where one returns Wasm from the first Wasm program which is then re-instantiated by the outer env. The other would be to pass in the api to create new Wasm envs over existing memory buffers.

See, https://copy.sh/v86/

MS-DOS, NES or C64 are useful for archival purposes because they are dead, frozen in time along with a large corpus of software. But there is a ton of complexity in implementing those systems with enough fidelity to run software.

Lua, Typed Assembly; https://en.wikipedia.org/wiki/Typed_assembly_language and Sector Lisp; https://github.com/jart/sectorlisp seem to have the right minimalism and compactness for archival purposes. Maybe it is sectorlisp+rv32+wasm.

If there are directions you would like Wasm to go, I really recommend attending the Wasm CG meetings.

https://github.com/WebAssembly/meetings

When it comes to an archival system, I'd like it to be able to run anything from an era, not just specially crafted binaries. I think Wasm meets that goal.

https://gist.github.com/dabeaz/7d8838b54dba5006c58a40fc28da9...

It allows WebAssembly to be programs which are run using a runtime on the command line, like Wasmtime, Wasmer, Wasm3, etc. Sometimes, you want to have a program which acts like a server, in that it can receive connections and send responses. This is what the patch for Tokio does.