Our great sponsors
-
murex
A smarter shell and scripting environment with advanced features designed for usability, safety and productivity (eg smarter DevOps tooling)
-
oil
Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
hush
hush (a Bourne-style shell) for the GNO multitasking environment on the Apple IIgs (by sheumann)
-
-
-
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
> I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell.
It's doable with existing tools. You have LSP to provide the syntactical framework and there's no shortage of alternatives to readline (I'd written my own[1] to use in murex[2], and open sourced that).
[1] https://github.com/lmorg/readline
[2] https://murex.rocks
The problem you still face is that a good shell will offer autocompletion suggestions for strings that aren't language keywords or function names. eg
- file names; and there's a lot of hidden logic in how to do this. Do you build in fzf-like support, just include fzf wholesale but increase your dependency tree, or go for basic path completion. Do you check metadata (eg hidden files and system files on Windows), include dot-prefixed files on Linux / UNIX, etc. How do you know when to return paths, or paths and files, or even know not to return disk items at all? (see next point)
- flags for existing CLI tools (assuming you want compatibility with existing tools). Fish and murex will parse man pages to populate suggestions, others rely entirely on the community to write autocompletion scripts.
- Are you including variables in your completion of strings. And if so are you reading the variables to spot if it's a path and then following that path. eg `cd $HOME/[tab]` should then return items inside a your home directory even though you've not actually specified your home directory as a string. That means the shell needs to expand the variables to see if it's a valid path. But that's a shell decision rather than a language feature.
Some of these lists might take a while to populate so you then have another problem. Do you delay the autocompletion list (bad UX because it slows the user down) or provide the autocompletion sooner. And if the latter, how do you do that without:
1. changing the items under what you're about to select causing you to accidentally select the wrong option
2. communicate that there are update clearly
3. ensure the UI is consistent when slower loading entries might not fit the same dimensions as the space allocated for the list (if you dynamically size your completions to fit the screen real estate)
4. ensure that there's still something present while you're lazy loading the rest of the suggestions; and that those early entries on the completion list are worthwhile and accurate
5. what about sorting the list? Alphabetical? By feature? etc
The REPL in murex was inspired by IDEs so I've spent a lot of time trying to consider how to provide the best UX around autocompletion. One thing I've learnt is that it's a lot harder to get right than it seems on the surface.
The language looks really quite interesting. I could see myself using it for quick scripts.
I think I'd prefer bash's noclobber behaviour to be the default redirection style. The explicitness of being forced to >| always feels like a nice safety feature to me, which would tie in nicely with their other defaults for safer scripting.
Also, not sure I'm keen on their minor change to the redirection syntax¹. It suggests that "2>1" and "2> 1" have very different results, which feels like an odd change to make when you're keeping the other aspects of that syntax. Probably says more about me though, I want 100% match or a complete break to force me to think.
I think it is interesting to compare the approach of oil², which has overlapping goals but is attacking the problem of improving shell scripting while attempting to keep the syntax.
¹ https://hush-shell.github.io/cmd/basic.html
² https://www.oilshell.org/
(note: if - like me - you're not seeing syntax highlighting for hush then enable JS)
The purpose of OP's project kind of reminded me of shell.js (shx) [1] which is a nodejs library that wraps all kinds of common UNIX commands to their own synchronously executed methods.
I guess that most shell projects start off as wanting to be a cross-platform solution to other operating systems, but somewhere in between either escalate to being their own programming language (like all the powershell revamps) or trying to reinvent the backwards-compatibility approach and/or POSIX standards (e.g. oil shell).
What I miss among all these new shell projects is a common standardization effort like sh/dash/bash/etc did back in the days. Without creating something like POSIX that also works on Windows and MacOS, all these shell efforts remain being only toy projects of developers without the possibility that they could actually replace the native shells of Linux distributions.
Most projects in the node.js area I've seen migrate their build scripts at some point to node.js, because maintaining packages and runtimes on Windows is a major shitshow. node.js has the benefit (compared to other environments) that it's a single .exe that you have to copy somewhere and then you're set to go.
When I compare that with python, for example, it is super hard to integrate. All the anaconda- or python-based bundles for ML engineers are pretty messed up environments on Windows; and nobody actually knows where their site-packages/libraries are really coming from and how to even update them correctly with upstream.
[1] https://github.com/shelljs/shelljs
I've already opened a pull request for the latter https://github.com/hush-shell/hush-shell.github.io/pull/1
Shellcheck is truly a game-changer, even for people who've been hacking Bash (etc) for 20+ years.
Side note: it's also one of (or the ?) most popular Haskell projects out there: https://github.com/koalaman/shellcheck/
Why use types? Why not introduce built in parsers to provide validations instead? (i.e. https://github.com/antlr/grammars-v4)
I think hush is going in the wrong direction. The majority of shell automation is associated with running IaC and container images / orchestration tools.
Shell scripts don't need to follow functional programming or OOP. They need to be container-oriented / VM-oriented / image-oriented with a conformation to YAML notation as this is already the adopted norm for containers and IaC.
This looks excellent - I outlined wanting something like a cross between Lua and Bash back in 2016, and at first glance, it would seem that this project by and large fits the bill: https://github.com/stuartpb/lash
Wasn't Hush already the name of the HUmbleSHell ( Hush is a Bourne/POSIX-style shell that was originally part of BusyBox ) ...https://github.com/sheumann/hush
https://xon.sh/ would make this so much better
> Any good text editor (e.g. vscode does it) should be able to translate aliases into long form commands.
It can be done with script Expand-Alias[1] that can be used even as CI/CD action so people don't have to think about this.
[1]: https://github.com/WormieCorp/Wormies-AU-Helpers/blob/develo...
> We need better shells.
Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs.
IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows.
I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history!
I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image at any time seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet...
Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary.
[1] https://github.com/piotrklibert/stshell
While being extremely small is a worthy goal, I suppose the aim of Hush is to make writing larger shell scripts easier and less error-prone. It's more for the niche of Perl of old than of minimal shells like ash.
For a very limited device, a very limited shell like that in Busybox is sufficient, because it likely does not need large shell scripts, or a lot of interactive work.
Looking at [1], current Hush is under 700k, which is still way smaller than Python or Perl, with much of its expressiveness.
[1]: https://github.com/hush-shell/hush/releases
> I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell.
It's doable with existing tools. You have LSP to provide the syntactical framework and there's no shortage of alternatives to readline (I'd written my own[1] to use in murex[2], and open sourced that).
[1] https://github.com/lmorg/readline
[2] https://murex.rocks
The problem you still face is that a good shell will offer autocompletion suggestions for strings that aren't language keywords or function names. eg
- file names; and there's a lot of hidden logic in how to do this. Do you build in fzf-like support, just include fzf wholesale but increase your dependency tree, or go for basic path completion. Do you check metadata (eg hidden files and system files on Windows), include dot-prefixed files on Linux / UNIX, etc. How do you know when to return paths, or paths and files, or even know not to return disk items at all? (see next point)
- flags for existing CLI tools (assuming you want compatibility with existing tools). Fish and murex will parse man pages to populate suggestions, others rely entirely on the community to write autocompletion scripts.
- Are you including variables in your completion of strings. And if so are you reading the variables to spot if it's a path and then following that path. eg `cd $HOME/[tab]` should then return items inside a your home directory even though you've not actually specified your home directory as a string. That means the shell needs to expand the variables to see if it's a valid path. But that's a shell decision rather than a language feature.
Some of these lists might take a while to populate so you then have another problem. Do you delay the autocompletion list (bad UX because it slows the user down) or provide the autocompletion sooner. And if the latter, how do you do that without:
1. changing the items under what you're about to select causing you to accidentally select the wrong option
2. communicate that there are update clearly
3. ensure the UI is consistent when slower loading entries might not fit the same dimensions as the space allocated for the list (if you dynamically size your completions to fit the screen real estate)
4. ensure that there's still something present while you're lazy loading the rest of the suggestions; and that those early entries on the completion list are worthwhile and accurate
5. what about sorting the list? Alphabetical? By feature? etc
The REPL in murex was inspired by IDEs so I've spent a lot of time trying to consider how to provide the best UX around autocompletion. One thing I've learnt is that it's a lot harder to get right than it seems on the surface.
Prepared a comparison to Next Generation Shell here:
https://github.com/ngs-lang/ngs/wiki/Hush-vs-NGS
On one hand we are on it since 2013 so there is "more" but I would actually like to highlight the difference in OOP and functional approaches: types, inheritance, multiple dispatch.