shelljs VS oil

When I need shell-like utilities from my JS scripts I've previously used shelljs [0]. It's neat that Bun is adding more built-in utilities though.

[0] https://github.com/shelljs/shelljs

Now we can open this new project and we're going to install one package, shelljs Shelljs is a great Command Line Utility for interacting with the command line in JavaScript.

Feels like this library is trying to solve a problem solved long ago by shelljs

The purpose of OP's project kind of reminded me of shell.js (shx) [1] which is a nodejs library that wraps all kinds of common UNIX commands to their own synchronously executed methods.

I guess that most shell projects start off as wanting to be a cross-platform solution to other operating systems, but somewhere in between either escalate to being their own programming language (like all the powershell revamps) or trying to reinvent the backwards-compatibility approach and/or POSIX standards (e.g. oil shell).

What I miss among all these new shell projects is a common standardization effort like sh/dash/bash/etc did back in the days. Without creating something like POSIX that also works on Windows and MacOS, all these shell efforts remain being only toy projects of developers without the possibility that they could actually replace the native shells of Linux distributions.

Most projects in the node.js area I've seen migrate their build scripts at some point to node.js, because maintaining packages and runtimes on Windows is a major shitshow. node.js has the benefit (compared to other environments) that it's a single .exe that you have to copy somewhere and then you're set to go.

When I compare that with python, for example, it is super hard to integrate. All the anaconda- or python-based bundles for ML engineers are pretty messed up environments on Windows; and nobody actually knows where their site-packages/libraries are really coming from and how to even update them correctly with upstream.

[1] https://github.com/shelljs/shelljs

`` When I then run this file with./bin/nodefile`, it exits, but the working directory of the current shell context has not changed. I have also tried shelljs, but that does not work either.

Oh people have tried - here are a few https://stackoverflow.com/questions/10239235/are-there-any-l...

I vaguely remember quite liking bish when I saw it years ago https://github.com/tdenniston/bish but it looks like no commits in 6 years.

This shelljs thing looks more promising, but really tedious to use https://github.com/shelljs/shelljs - shell.rm('-rf', 'out/Release'); I'd rather suffer proper bash than have to do that sort of thing.

Nothing seems to have really caught on so far. Bash is easy to learn and hack on, and before you know it, that simple install.sh that started out moving a few files around is 5000 lines, unmaintainable, and critical to bootstrapping your software :)

I personally prefer shelljs for stuff like this. zx seems pretty high on the "insane syntactic sugar" train.

As we are creating starter files, let's use ShellJS to run commands like git clone, mkdir...

This talks a bunch about the "good run" of current scripting languages, including for example JavaScript.

But JavaScript, as an actual scripting language, has been pretty primitive but finally starting to become a real candidate for actual scripting. There's imo crufty not very great options like shelljs[1]. But adding a tagged-template string for system(), for calling things, and a little bit of standard library has made JS a much more interesting & competent scripting language. Those efforts are being done in ZX[2].

I like the idea of the topic, exploring it. But the author feels off in a number of places.

> What TypeScript showed is that you could join together the idea of a flexible lightweight (and optional!) type system onto an existing programming language, and do so successfully. . . .The question then is - what if you created a programming language from the start to have this kind of support?

Personally I just don't think languages matter very much. They're very similar, by & large. They have different tooling, packaging, somewhat different looks/feels for executing code, and their standard libraries are different. But TypeScript is popular & fast at least 90% because it is JS, because it works with JS things. Arguing that we should try to recreate TypeScript apart from JS sounds like a mind blowing waste of time. Also, Deno has good integrated TypeScript support.

On the topic of easy parallelism, JavaScript promises are imo quite easy to stitch together & use & quite available.

One of the main issues I see with easy-parallelism is that it's too easy: there's too many cases for uncontrolled parallelism. Throwing tarn.js or other worker-pools at problems seems all too common. But one is still left stitching together each pool/stage of work. I'd like to see SEDA[3] like architectures emerge, and perhaps get added to something like ZX standard library.

[1] https://github.com/shelljs/shelljs

[2] https://github.com/google/zx

[3] https://en.wikipedia.org/wiki/Staged_event-driven_architectu...

will prevent almost all of the "silent footguns".

YSH has strict:all and then a bunch of NEW features.

There's been good feedback recently, which has led to many concrete changes. So your experience can definitely influence the language! https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback

Regarding writing tools/tests/benchmarks in bash+Python, vs. writing tools in your main language:

I think we might eventually concede that something Debian-like is the “standard development environment” (at least for server side stuff, i.e. not iOS apps)

In this case, bash+Python is a non-issue. It works extremely reliably. That’s actually why I use it! Everything else seems to break, or it’s really slow (node.js is a very common alternative).

- Microsoft conceded this back in ~2017, by building Linux into their kernel with WSL, and providing Ubuntu on top

Yes bash + Python is a disaster on Windows (I have scars from it), but Microsoft agrees that the right place to solve that is in Windows :-)

- Every CI system runs Debian/Ubuntu

- Every hosting provider runs Debian/Ubuntu

- Every online dev env like gitpod.io provides Debian/Ubuntu

This is somewhat related to remote dev envs: https://lobste.rs/s/ucirlx/lapdev_self_hosted_remote_dev

One vision for https://www.oilshell.org/ is that the CI environment is the dev environment is the hosting environment.

Everything is just an equal node in a distributed system. BUT it’s more git like, in that you explicitly sync and work “locally”, wherever that is. You don’t have the network chatter and flakiness of “the cloud”.

Oils has a very large set of monotonically increasing properties too - https://www.oilshell.org/release/0.21.0/quality.html

All that is bash+Python that is run on every commit, and it’s extremely good at catching bugs and perf regressions.

I’m skeptical that any project has that level of quality automation written in pure Rust or Zig. More likely it’s a bunch of cloud services with YAML.

Also a bunch of “hard-coded” toolchains that you can’t script with bespoke code. Like some shell commands in your package.json, which is just a worse way of writing a shell script.

Our quality process is all self-hosted, in the repo, and runs on both Github Actions and sourcehut - https://www.oilshell.org/release/0.21.0/pub/metrics.wwz/line...

bash and Python runs perfectly on Github Actions and sourcehut, with zero change. Containers also do.

(Although we need to unify the CI and release, because the release runs on 2 different real hardware machines, while CI is cloud only.)

Also, a main point Oils is that bash now has another highly compatible, spec-driven implementation – OSH. Having 2 independent implementations is something newer languages don’t have.

(copy of lobste.rs comment)

in your bashrc to enable it. I've used it for probably ~18 years now.

It also works with https://www.oilshell.org/ since we use GNU readline. Just 'set -o vi' in ~/.config/oils/oshrc

No other shell does that.

But I didn't know it was called MULTIOS until now. (I guess that's read "mult I/O's"? I have a hard time not reading it was multi-OS :) )

It seems a bit niche to be honest, but it's possible to support in Oils.

---

Oils also uses Unix domain sockets already for the headless shell protocol

https://github.com/oilshell/oil/wiki/Headless-Mode

We could do something like dgsh, but so far I haven't seen a lot of uptake / demand. Every time it's mentioned, somebody kinda wants it, and then it kinda peters out again ... still possible though.

I think flat files work fine for a lot of use cases, and once you add streaming, you also want monitoring, more control over backpressure/queue sizes, etc.

which works well. You don't have to clean when rebuilding variants. IMO this is 100% essential for writing C++ these days. You need a bunch of test binaries, and all tests should be run with ASAN and UBSAN.

---

I wrote a mini-bazel on top of Ninja with these features:

https://www.oilshell.org/blog/2022/10/garbage-collector.html...

So it's ~1700 lines, but for that you get the build macros like asdl_library() generating C++ and Python (the same as proto_library(), a schema language that generates code)

And it also correctly finds dependencies of code generators. So if you change a .py file that is imported by another .py file that is used to generated a C++ header, everything will work. That was one of the trickier bits, with Ninja implicit dependencies.

I also use the Bazel-target syntax like //core/process

This build file example mixes low level Ninja n.rule() and n.build() with high level r.cc_library() and so forth. I find this layering really does make it scale better for bigger projects

https://github.com/oilshell/oil/blob/master/asdl/NINJA_subgr...

Some more description - https://lobste.rs/s/qnb7xt/ninja_is_enough_build_system#c_tu...

This is sort of a category error...

re2c is a lexer generator, and YAML and Python are recursive/nested formats.

You can definitely use re2c to lex them, but it's not the whole solution.

I use it for everything possible in https://www.oilshell.org, and it's amazing. It really reduces the amount of fiddly C code you need to parse languages, and it drops in anywhere.

SEEKING VOLUNTEERS - https://www.oilshell.org/ - https://github.com/oilshell/oil/

I'm looking for people to help fill out the "standard library" for Oils/YSH. We're implementing a shell for Python and JavaScript programmers who avoid shell!

On the surface, this is writing some very simple functions in typed Python. But I've realized that the hardest parts are specifying, TESTING, and documenting what the functions do.

---

The most recent release announcement also asks for help - https://www.oilshell.org/blog/2024/01/release-0.19.0.html (long)

If you find all those details interesting (if maybe overwhelming), you might have a mind for language design, and could be a good person to help.

Surveying what Python and JavaScript do is very helpful, e.g. for the recent Str.replace() function, which is nontrivial (takes a regex or string, replacement template or string)

But there are also very simple methods to get started, like Dict.values() and List.indexOf(). Other people have already contributed code. Examples:

https://github.com/oilshell/oil/commit/58d847008427dba2e60fe...

https://github.com/oilshell/oil/commit/8f38ee36d01162593e935...

This can also be useful to tell if you'll have fun working on the project - https://github.com/oilshell/oil/wiki/Where-Contributors-Have...

More on #help-wanted on Zulip (requires login) - https://oilshell.zulipchat.com/#narrow/stream/417617-help-wa...

Please send a message on Github or Zulip! Or e-mail me andy at oilshell dot org.

This is true, but then the corrolary is that new PRs need to come with this higher and rigorous level of test coverage.

And then that becomes a bit of a barrier to contribution -- that's a harness

I often write entirely new test harnesses for features, e.g. for https://www.oilshell.org, many of them linked here . All of these run in the CI - https://www.oilshell.org/release/latest/quality.html

The good thing is that it definitely helps me accept PRs faster. Current contributors are good at this kind of exhaustive testing, but many PRs aren't