zig VS oils

Finished up to 109.

Kinda sad that async is broken and there's a huge pile of gatekeeping requirements against bringing it back. https://github.com/ziglang/zig/wiki/FAQ#what-is-the-status-o...

Actually, nobody.

Here is the commit where Reader/Writer was introduced: https://github.com/ziglang/zig/commit/5e212db29cf9e2c06aba36...

This is a few months after `git init`. You can see I was really just working on the parser, with a toy example to get things started.

Over time, I merged contributions that made minor changes and shuffled things around, and these APIs evolved to kind of work okay. But nobody really considered "the Zig IO stack" as a whole and put in design effort. That is happening for the first time right now.

This list's Zig as an entry, despite the Zig project having very clear plans[0] for a 1.0 release. That's not 0ver, it's just the beta stage of semver.

[0] https://github.com/ziglang/zig/milestone/2

I usually structure teaching the same way done in https://www.writethedocs.org/videos/eu/2017/the-four-kinds-o.... So "the Quick Walkthrough Guide will explain what dk scripts are and give you small examples to run" is simply a learning-oriented tutorial which is mostly about giving students confidence and visual feedback. And simultaneously it an explanation of nothing (the video has a great explanation for why to do that). So, I agree that an explanation of threads + Internet + cross-compilation would quite nuts, but for an experienced developer I'd expect to see a meaty example (take a look at https://ziglang.org/ for comparison).

One concrete action may be to make two distinct Quick Start guides ... one for the experienced and one for the inexperienced students though. Is that your thinking?

> I feel like I can write powerful code in any language, but the goal is to write code for a framework that is most future proof, so that you can maintain modular stuff for decades.

I like Zig a lot, but long-term maintainability and modularity is one of its weakest points IMHO.

Zig is hostile to encapsulation. You cannot make struct members private: https://github.com/ziglang/zig/issues/9909#issuecomment-9426...

Key quote:

> The idea of private fields and getter/setter methods was popularized by Java, but it is an anti-pattern. Fields are there; they exist. They are the data that underpins any abstraction. My recommendation is to name fields carefully and leave them as part of the public API, carefully documenting what they do.

You cannot reasonably form API contracts (which are the foundation of software modularity) unless you can hide the internal representation. You need to be able to change the internal representation without breaking users. I hope Zig reverses this decision someday and supports private fields.

> Did Zig really do that?

https://github.com/ziglang/zig/issues/544#issuecomment-61807...

> In order to simplify everything, tabs are not allowed. Spaces are necessary; we can't ban spaces.

They seem to be open to the alternative but not to a solution that isn't forced on people:

> Maybe someday, we'll switch to tabs for indentation, spaces for alignment and make it a compile error if they are incorrectly mixed

Zig already has a builtin code formatter that automatically changes formatting to their preference, but that's not enough.

Zig solved this by not having a string type at all and not shipping full Unicode support in std: https://github.com/ziglang/zig/issues/234#issuecomment-27630....

I'll keep an eye out for the blog post!

We're still working (intermittently) on a better shell <-> terminal interface. We actually did the first demo back in 2021, before we became aware of Warp. And here is an update from a month ago, which mentions Warp / IntelliJ / VSCode:

https://lobste.rs/s/viwe6p/open_source_can_t_coordinate#c_px...

That is, we did some demos with a nushell contributor, to see if there is a protocol that Oils and nushell can agree on. The big constraints seems to be bash and zsh. (I still believe in graceful evolution, not building a "inner platform")

I think these 2025 demos went pretty well, but it's still in the demo/research phase.

After working in this area for many years, I think the bottleneck is coordination, mentioned here - https://lobste.rs/s/ejcoud/dreaming_smel_shell

i.e. there are many separate projects doing similar things, and most of them get to 80% or so. I like that you mention a lot of prior art!

More links: https://github.com/oils-for-unix/oils/wiki/Headless-Shell-Pr...

Feel free to join any of the Zulip discussions ... there are a bunch of prototypes floating around, but nothing set in stone!

JSON is not a friendly format to the Unix shell — it’s hierarchical, and cannot be reasonably split on any character

Yes, shell is definitely too weak for JSON parsing!

I would also argue that Awk is too weak

The following code assumes that it will be fed valid JSON. It has some basic validation as a function of the parsing and will most likely throw an error if it encounters something strange, but there are no guarantees beyond that.

Yeah I don't like that! If you don't reject invalid input, you're not really parsing

---

OSH and YSH (https://oils.pub) both have JSON built-in, and they have the hierarchical/recursive data structures you need for the common Python/JS-like API:

    osh-0.33$ var d = { date: $(date --iso-8601) }

Wow great bug!

> Bash forgot to reset errno before the call. For about 30 years, no one noticed

I have to say, this part of the POSIX API is maddening!

99% of the time, you don't need to set errno = 0 before making a call. You check for a non-zero return, and only then look at errno.

But SOMETIMES you need to set errno = 0, because in this case readdir() returns NULL on both error and EOF.

I actually didn't realize this before working on https://oils.pub/

---

And it should go without saying: Oils simply uses libc getcwd() - we don't need to support system with a broken getcwd()!

Although a funny thing is that I just fixed a bug related to KSH that AT&T ksh (the original shell, that bash is based on!) hasn't fixed for 30+ years too!

https://www.illumos.org/issues/17442

https://github.com/oils-for-unix/oils/issues/2058

There is subtle issue that with respect to:

1) "trusting" the $PWD value you inherit from another process

2) Respecting symlinks - why you can't just call getcwd() !

    if (*p != '/' || stat(p, &st1) || stat(".", &st2) ||

Not sure if others had that same experience. I think it can't really be accurate because it doesn't have a lot of build time info. So I think they could have embraced "coarseness" more, to make it faster

Although maybe I am confusing the UI speed with the analysis speed. (One reason that this was originally a Codeberg link is that Codeberg/Forejo's UI is faster, without all the nav stuff)

There are some related links here, like How To Build Static Analyzers in Orders of Magnitude Less Code:

https://github.com/oils-for-unix/oils/wiki/Polyglot-Language...

https://oils.pub/

There's probably more.

The shell has a ____wide____ userbase with many kinds of users. Depending on your goal, the rabbit hole can go very deep (how portable across interpreters, how dependant on other binaries, how early can it work in a bootstrap scenario, etc).

These are mine:

https://github.com/alganet/coral

I am not sure if this helps or not, but if you want a bash-compatible shell, translating Oils to Ada or porting it to Windows could be easier than adding bash arrays.

It is extremely hairy, and it differs from bash version to version

https://news.ycombinator.com/item?id=43910883

I mentioned in another comment that Koichi Murase, who is a bash contributor, and wrote the largest shell program in the world, just overhauled the bash array support in OSH

Relevant tests -- he added a huge number recently, and made them work:

https://oils.pub/release/0.29.0/test/spec.wwz/osh-py/array-s...

https://oils.pub/release/0.29.0/test/spec.wwz/osh-py/array-a...

https://oils.pub/release/0.29.0/test/spec.wwz/osh-py/ble-idi...

Our tests are thorough enough that we ROUTINELY find bugs in bash, like integer overflow bugs.

Koichi also knows about the differences between say bash 4.3, 4.4, 5.0, 5.1, etc. Because he has a very large program that uses bash arrays all over the place.

---

Feel free to use our tests in any case (other shells like the Scheme shell used to bootstrap Guix have)

And feel free to post a message to https://github.com/oils-for-unix/oils if you're interested or have questions

Hm I also made a taxonomy of 5 categories of config languages, which is a bit different

Survey of Config Languages https://github.com/oils-for-unix/oils/wiki/Survey-of-Config-...

    Languages for String Data

- and we have built-in JSON, etc.

This is a hard design challenge, but I just made a release with an overhaul of Hay - https://oils.pub/release/0.28.0/

Hay version 1 was hard-coded in the interpreter - https://oils.pub/release/0.28.0/doc/hay.html

But we realized it's actually better to *self-host* it in YSH, using YSH reflection. We will be testing this by rewriting Hay in YSH

---

So that's our language design response to https://news.ycombinator.com/item?id=43386115

> It's madness that languages are effectively siloed from each other.

Instead of tiers 4, 5, 6 being silo'd, we have them all under YSH and the Oils runtime (which is tiny, 2.3 MB of pure native code).

(As a bonus, OSH also runs on the Oils runtime, and it's the most bash-compatible shell!)

----

[1] Garbage Collection Makes YSH Different - https://www.oilshell.org/blog/2024/09/gc.html

Shell, Awk, and Make Should Be Combined - https://www.oilshell.org/blog/2016/11/13.html - all these languages lack GC!

[2] Survey of Config Languages - https://github.com/oils-for-unix/oils/wiki/Survey-of-Config-... - divides this category into 5 tiers:

    1. Languages for String Data

Looking at the original posting, the maintainer knows what the bug is. The stopper is more along the lines of "This looks synthetic, so it's not high priority. If it's a priority for you, then you should fix it yourself or make an argument for its priority." followed by oguz saying "It's not actually my problem either, I'm not invested in osh.". (not actual quotes: https://github.com/oils-for-unix/oils/issues/1881#issuecomme...)

The bug is that osh leaves the script fd open for the child process when executing a command, allowing the child command to mess with the parent script. I don't see this as a huge issue for everyday interactive use, but I'd prefer my shell to not do that. If I were executing programs with limited capabilities, I'd consider this a possible vector for security bugs. That is to say, I wouldn't use osh scrpts if I cared about security.

Congrats to the fish team! Great writeup with lots of interesting detail.

I wonder if this is the biggest project that has moved from C++ entirely to Rust (or maybe even C to Rust?) It probably has useful lessons for other projects.

If I'm reading this right, it looks like fish was not released as a hybrid C++ / Rust program, with the autocxx-generated bindings. There was a release during that time, but it says "fish 3.7 remains a C++ program" [1]

It sounds like they could have released if they wanted to, but there was a last stage of testing that didn't happen until the end.

Some people didn't quite get the motivation for adding C++ features to Rust [2], and vice versa, to enable inter-op. But perhaps this is a good case study. It would be nice if you could could just write new Rust code in a C++ codebase, without writing/generating bindings, and then throwing them away, which is mentioned in this post.

--

Also the #1 gripe with Rust seems to be that it supports version detection, not feature detection.

But feature detection is better for distros, web browsers, and compilers:

https://github.com/oils-for-unix/oils/wiki/Feature-Detection...

Version detection is why Chrome and IE pretend to be Mozilla, and why Clang pretends to be GCC. Feature detection doesn't cause this problem!

[1] https://github.com/fish-shell/fish-shell/releases

[2] e.g. https://news.ycombinator.com/from?site=safecpp.org