RVS_BlueThoth VS oil

I generally have a “What Problem Does This Solve?” section in my READMEs.

https://github.com/LittleGreenViper/LGV_TZ_Lookup#what-probl...

https://github.com/LittleGreenViper/LGV_MeetingServer#what-p...

https://github.com/RiftValleySoftware/RVS_Spinner#what-probl...

https://github.com/RiftValleySoftware/RVS_BlueThoth#what-pro...

https://github.com/RiftValleySoftware/RVS_PersistentPrefs#wh...

etc.

> I had one developer take 6 months to build a (relatively simple) top nav for a web app. This shouldn't have taken more than 1-2 weeks, even with a careful eye for detail.

Oh, you mean "bikeshedding."

Here's an example of the difference between basic quality, and High Quality:

If you look at most of the repos for SPM modules in my portfolio[0], you'll see that the vast majority have test harnesses. I prefer using test harnesses[1].

These test harnesses tend to be pretty damn robust apps. Many are "ready for app store" robust. A lot of folks would just publish them, "as is." I've been writing apps for a very long time. I'm fairly good at this.

I can write a fairly good test harness, with full app capabilities, in less than a day. If I take the time to localize it, maybe add a day or so.

Here's an example of some test harnesses[2]. Note that there are four of them. These represent the four different target environments for Apple (iOS/iPadOS, WatchOS, TVOS, and MacOS). I'll probably need to fork iOS and iPadOS, in the future, but we're not there, yet. A single codebase is still good for both.

They test a Bluetooth framework[3].

It probably took me around a week or so, to write each one. They are pretty damn good. I think they are all "App Store ready."

I decided to actually go ahead, and create a set of apps, based on these[4], [5], [6].

I spent well over a month, on each, after merging over the test harness codebases, to make them ready for the App Store. Lots of UX testing, removing code that only applied to testing, and adding "friendlier" user interface.

I'm working on an app that I started about a year ago. Actually, I started it over ten years ago, if you include the two servers that I wrote, upon which it depends.

One of the reasons that it has taken so long, is that I have truncated months of work, and tossed them in the garbage, because they were not the proper way to go. I have an "evolutionary design" process[7], that means this can happen. I plan for it. I've probably shitcanned three months' of work.

Another thing that I do, is have an "always beta" approach to Quality. I maintain the product at "incomplete, but ship Quality" status for as much of the project as possible. In fact, I've been sharing it with the team, using TestFlight, since Oct 3, 2020 at 7:47 AM (I got that from the TestFlight metadata).

That means that the app has been stable and robust enough for user testing, and approval for basic App Store release (TestFlight External Testing is a more relaxed standard, but try pushing out a crasher, and see how far that goes).

I add localization support, accessibility, Dark Mode support, leak testing, etc., at every turn. It's very useful, because I can solicit immediate feedback from non-tech team members. It also means that the "basics" for App Store release are constantly being tested and validated.

Even more useful, if we want to ask for money, it's dam easy. We just loop the person we're begging from, into the TestFlight External Tester pool, and they can run the app without a Marketing chaperone, or sacrifices to the demo gods. We can also get valuable feedback from them.

It's really, really nice, and it has been, for many months.

I feel like we are now at a "starting point." Even though it has been a fully-functioning, release-ready app for the last couple of months, it need the "MVP treatment," where the testing pool is expanded, and we start applying it to "in the wild" scenarios.

Lots of companies use their customers as guinea pigs for the first several releases; usually by shoving baling-wire-and-duct-tape junk down their throats (and making them pay for it), before hitting their stride. It's a deliberate strategy. Some months ago, I read a post, here by a founder, declaring that "if you don't get physically sick at the quality of the code in your MVP, then you are spending too much time on the code quality."

Basically, deliberately write garbage, and force it on your users.

One of the reasons that I took on this project, was the founder is a friend of mine. He is running it as an NPO (501c3), and putting his own money into it. He doesn't really have much of it, to begin with. Also, more alarmingly, he didn't actually have a particularly good idea of what, exactly, he wanted the app to be. That's a recipe for disaster.

He asked me to help him vet some development shops he was approaching, to realize his vision.

It was eye-opening. He got a number of ridiculous quotes. I know what is necessary for this type of project (not small). For example, when one said that they'll deliver a full multi-server, multi-client app for MVP in three months (firm), upon getting a vague, hand-wavy requirements spec, it was hard for me to keep a straight face.

After a few of these, I just got disgusted, and said "Screw this. I'll do it." I've been developing it for free, as a native iOS/iPadOS app.

He has to pinch himself.

[0] https://stackoverflow.com/story/chrismarshall

[1] https://littlegreenviper.com/miscellany/testing-harness-vs-u...

[2] https://github.com/RiftValleySoftware/RVS_BlueThoth/tree/mas...

[3] https://github.com/RiftValleySoftware/RVS_BlueThoth

[4] https://apps.apple.com/us/app/blue-van-clef-for-mobile/id151... (iOS -Includes Watch app)

[5] https://apps.apple.com/us/app/blue-van-clef/id1529005127 (Mac)

[6] https://apps.apple.com/us/app/blue-van-clef-for-tv/id1529181... (TV)

[7] https://littlegreenviper.com/miscellany/evolutionary-design-...

Some related references (on a somewhat messy wiki page) - https://github.com/oilshell/oil/wiki/Compact-AST-Representat...

Feel free to add others

will prevent almost all of the "silent footguns".

YSH has strict:all and then a bunch of NEW features.

There's been good feedback recently, which has led to many concrete changes. So your experience can definitely influence the language! https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback

Regarding writing tools/tests/benchmarks in bash+Python, vs. writing tools in your main language:

I think we might eventually concede that something Debian-like is the “standard development environment” (at least for server side stuff, i.e. not iOS apps)

In this case, bash+Python is a non-issue. It works extremely reliably. That’s actually why I use it! Everything else seems to break, or it’s really slow (node.js is a very common alternative).

- Microsoft conceded this back in ~2017, by building Linux into their kernel with WSL, and providing Ubuntu on top

Yes bash + Python is a disaster on Windows (I have scars from it), but Microsoft agrees that the right place to solve that is in Windows :-)

- Every CI system runs Debian/Ubuntu

- Every hosting provider runs Debian/Ubuntu

- Every online dev env like gitpod.io provides Debian/Ubuntu

This is somewhat related to remote dev envs: https://lobste.rs/s/ucirlx/lapdev_self_hosted_remote_dev

One vision for https://www.oilshell.org/ is that the CI environment is the dev environment is the hosting environment.

Everything is just an equal node in a distributed system. BUT it’s more git like, in that you explicitly sync and work “locally”, wherever that is. You don’t have the network chatter and flakiness of “the cloud”.

Oils has a very large set of monotonically increasing properties too - https://www.oilshell.org/release/0.21.0/quality.html

All that is bash+Python that is run on every commit, and it’s extremely good at catching bugs and perf regressions.

I’m skeptical that any project has that level of quality automation written in pure Rust or Zig. More likely it’s a bunch of cloud services with YAML.

Also a bunch of “hard-coded” toolchains that you can’t script with bespoke code. Like some shell commands in your package.json, which is just a worse way of writing a shell script.

Our quality process is all self-hosted, in the repo, and runs on both Github Actions and sourcehut - https://www.oilshell.org/release/0.21.0/pub/metrics.wwz/line...

bash and Python runs perfectly on Github Actions and sourcehut, with zero change. Containers also do.

(Although we need to unify the CI and release, because the release runs on 2 different real hardware machines, while CI is cloud only.)

Also, a main point Oils is that bash now has another highly compatible, spec-driven implementation – OSH. Having 2 independent implementations is something newer languages don’t have.

(copy of lobste.rs comment)

in your bashrc to enable it. I've used it for probably ~18 years now.

It also works with https://www.oilshell.org/ since we use GNU readline. Just 'set -o vi' in ~/.config/oils/oshrc

No other shell does that.

But I didn't know it was called MULTIOS until now. (I guess that's read "mult I/O's"? I have a hard time not reading it was multi-OS :) )

It seems a bit niche to be honest, but it's possible to support in Oils.

---

Oils also uses Unix domain sockets already for the headless shell protocol

https://github.com/oilshell/oil/wiki/Headless-Mode

We could do something like dgsh, but so far I haven't seen a lot of uptake / demand. Every time it's mentioned, somebody kinda wants it, and then it kinda peters out again ... still possible though.

I think flat files work fine for a lot of use cases, and once you add streaming, you also want monitoring, more control over backpressure/queue sizes, etc.

which works well. You don't have to clean when rebuilding variants. IMO this is 100% essential for writing C++ these days. You need a bunch of test binaries, and all tests should be run with ASAN and UBSAN.

---

I wrote a mini-bazel on top of Ninja with these features:

https://www.oilshell.org/blog/2022/10/garbage-collector.html...

So it's ~1700 lines, but for that you get the build macros like asdl_library() generating C++ and Python (the same as proto_library(), a schema language that generates code)

And it also correctly finds dependencies of code generators. So if you change a .py file that is imported by another .py file that is used to generated a C++ header, everything will work. That was one of the trickier bits, with Ninja implicit dependencies.

I also use the Bazel-target syntax like //core/process

This build file example mixes low level Ninja n.rule() and n.build() with high level r.cc_library() and so forth. I find this layering really does make it scale better for bigger projects

https://github.com/oilshell/oil/blob/master/asdl/NINJA_subgr...

Some more description - https://lobste.rs/s/qnb7xt/ninja_is_enough_build_system#c_tu...

This is sort of a category error...

re2c is a lexer generator, and YAML and Python are recursive/nested formats.

You can definitely use re2c to lex them, but it's not the whole solution.

I use it for everything possible in https://www.oilshell.org, and it's amazing. It really reduces the amount of fiddly C code you need to parse languages, and it drops in anywhere.

SEEKING VOLUNTEERS - https://www.oilshell.org/ - https://github.com/oilshell/oil/

I'm looking for people to help fill out the "standard library" for Oils/YSH. We're implementing a shell for Python and JavaScript programmers who avoid shell!

On the surface, this is writing some very simple functions in typed Python. But I've realized that the hardest parts are specifying, TESTING, and documenting what the functions do.

---

The most recent release announcement also asks for help - https://www.oilshell.org/blog/2024/01/release-0.19.0.html (long)

If you find all those details interesting (if maybe overwhelming), you might have a mind for language design, and could be a good person to help.

Surveying what Python and JavaScript do is very helpful, e.g. for the recent Str.replace() function, which is nontrivial (takes a regex or string, replacement template or string)

But there are also very simple methods to get started, like Dict.values() and List.indexOf(). Other people have already contributed code. Examples:

https://github.com/oilshell/oil/commit/58d847008427dba2e60fe...

https://github.com/oilshell/oil/commit/8f38ee36d01162593e935...

This can also be useful to tell if you'll have fun working on the project - https://github.com/oilshell/oil/wiki/Where-Contributors-Have...

More on #help-wanted on Zulip (requires login) - https://oilshell.zulipchat.com/#narrow/stream/417617-help-wa...

Please send a message on Github or Zulip! Or e-mail me andy at oilshell dot org.

This is true, but then the corrolary is that new PRs need to come with this higher and rigorous level of test coverage.

And then that becomes a bit of a barrier to contribution -- that's a harness

I often write entirely new test harnesses for features, e.g. for https://www.oilshell.org, many of them linked here . All of these run in the CI - https://www.oilshell.org/release/latest/quality.html

The good thing is that it definitely helps me accept PRs faster. Current contributors are good at this kind of exhaustive testing, but many PRs aren't