stshell VS readline

It's slowly getting traction - Kotlin on Android has a "live update" feature (in development, only available in alpha release), for example. Multiple less mainstream languages also offer the feature. Nim got it in the last major release, for example. V has it as one of the base features. Erlang and Elixir had it since forever. Common Lisp as well. Racket and Clojure are a little more limited than CL, but also support it. Many interpreted languages offer some degree of this, either by default (JavaScript) or as a library/package (Python, Ruby).

In general, programming language features take about 20 to 30 years to go from obscure niche implementation into the mainstream. Look at lambdas - anonymous function literals - they're now everywhere, including Java and C++. Ten years ago, though, only some scripting languages had it. The feature itself is as old as the bones of the Earth (LISP, 1960, 63 years old). The same is true for many other "advanced" features. I think this is tied to generational changes - each generation of programmers has a chance to bring one or two lesser known features into mainstream, and then they're content with that. Other features have to wait for the next generation to discover them.

As for Smalltalk - I made a mistake and based the implementation on GNU Smalltalk, which is unmaintained. I should have gone with Smalltalk/X, Visual Works, or (begrudgingly) Pharo (or Cuis). I started the project as yet another attempt at making a MUD, but then changed focus to making a productive command-line-based programming env for Smalltalk. Then I changed my mind again and tried to make it into a usable shell. Here's the project: https://github.com/piotrklibert/stshell/ The screenshots focus on the REPL/shell side, but in the source you'll see things like "server", "player", and "world". There are a few locations IIRC and you can move your character between them still. It was an interesting project, but without a clear vision of what it should be it lost focus and I left it to rot after a while :(

> We need better shells.

Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs.

IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows.

I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history!

I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image at any time seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet...

Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary.

[1] https://github.com/piotrklibert/stshell

... I kind of am, though. Which is why I didn't know what to do. I don't see a lot of free open source projects with extensive documentation in their README. Some, yes, but for example here's the readline library I'm using. I, in my well-meaning ignorance, supplied 50 pages of documentation and people are behaving like I ate a baby 'cos it's in the wrong format. I will now put it all in the README as people would like, but I did genuinely act out of ignorance and not out of a wish to insult the customs of the tribe.

> I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell.

It's doable with existing tools. You have LSP to provide the syntactical framework and there's no shortage of alternatives to readline (I'd written my own[1] to use in murex[2], and open sourced that).

[1] https://github.com/lmorg/readline

[2] https://murex.rocks

The problem you still face is that a good shell will offer autocompletion suggestions for strings that aren't language keywords or function names. eg

- file names; and there's a lot of hidden logic in how to do this. Do you build in fzf-like support, just include fzf wholesale but increase your dependency tree, or go for basic path completion. Do you check metadata (eg hidden files and system files on Windows), include dot-prefixed files on Linux / UNIX, etc. How do you know when to return paths, or paths and files, or even know not to return disk items at all? (see next point)

- flags for existing CLI tools (assuming you want compatibility with existing tools). Fish and murex will parse man pages to populate suggestions, others rely entirely on the community to write autocompletion scripts.

- Are you including variables in your completion of strings. And if so are you reading the variables to spot if it's a path and then following that path. eg `cd $HOME/[tab]` should then return items inside a your home directory even though you've not actually specified your home directory as a string. That means the shell needs to expand the variables to see if it's a valid path. But that's a shell decision rather than a language feature.

Some of these lists might take a while to populate so you then have another problem. Do you delay the autocompletion list (bad UX because it slows the user down) or provide the autocompletion sooner. And if the latter, how do you do that without:

1. changing the items under what you're about to select causing you to accidentally select the wrong option

2. communicate that there are update clearly

3. ensure the UI is consistent when slower loading entries might not fit the same dimensions as the space allocated for the list (if you dynamically size your completions to fit the screen real estate)

4. ensure that there's still something present while you're lazy loading the rest of the suggestions; and that those early entries on the completion list are worthwhile and accurate

5. what about sorting the list? Alphabetical? By feature? etc

The REPL in murex was inspired by IDEs so I've spent a lot of time trying to consider how to provide the best UX around autocompletion. One thing I've learnt is that it's a lot harder to get right than it seems on the surface.