KaithemAutomation VS scheme-for-max

My SD protection script, a few months old and may need some updates since a lot seems to have changed in Pi OS:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Current version doesn't disable swap, that's in a separate optional file, but the next version will.

It can't be done in a one size fits all script unless you're launching chromium the same way, but do something like:

I don't usually do full read only, what I'll do is run a script that turns off stuff that does not need to be writing to the disk all the time.

Unfortunately, some software is database-oriented and likes to write to disk for every tiny thing, so the approach doesn't work with stuff like Home Assistant unless you carefully configure logging.

The basic simple stuff doesn't really cause any user-level noticable changes:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

After that, I disable and mask apt-daily (The Debian auto updater), and purge dphys-swapfile.

My full set of assorted tweaks can be found here, some might not be relevant for you:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Next, I often run Chromium as a kiosk, and Chromium likes to hammer the SD card, so I set the XDG folder environment variables to make it put it's stuff in RAM. My embedded chrome stuff can be found here:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Things like that are easy enough to set up on Raspbian with a script. You also need a bunch of tmpfs mounts, including some in crazy places(Regular Pi has a log file under ~/.cache that can fill disks and crash servers, and if you don't fix it Chromium writes seemingly useless crap constantly. There's auto updates on some systems, which is terrible if you're on a private WiFi, not doing internet stuff.

I'm not using zram at the moment, just getting rid of swap, but my current script to get a Pi ready for embedded projects is here: https://github.com/EternityForest/KaithemAutomation/blob/mas...

The only FOSS thing I've done that I think is really worth telling people about is KaithemAutomation, a home automation server in pure Python with a bit easier setup than Home Assistant, and some features aimed at commercial installs like room escape control, and some pretty decent network video recorder features.

https://github.com/EternityForest/KaithemAutomation

I put 6 years or so into it, and have used it on plenty of contract projects, but so far I don't think anyone else is interested.

Possibly because it's largely UI and CRUD over existing functionality, and there's not much particularly exciting to the hacker community, few interesting algorithms, it's not minimalist at all, etc.

Plus it has a lot of dependencies that might or might not exist outside of Debian, I've never looked into how it would run on the more DIY distros since I've never used them.

Then after that I square every pixel and square root the mean of the whole image.

I forgot how complicated this was and how many tweaks I added!

Code here: https://github.com/EternityForest/KaithemAutomation/blob/mas...

And for the motion detector specifically:

https://github.com/EternityForest/KaithemAutomation/blob/9db...

pure Python and heavily built around runtime modification(https://github.com/EternityForest/KaithemAutomation), and is designed to run for months, although it's not meant to be modified literally while running production(Not that that stops us from making last minute fixes...).

The big problem with it in Python is state. If you have a reference to something, and that something changes, and you have a reference to the old one, that should be a singleton, you are probably in an unhappy position.

The only way it can work in real life(In Python at least) is if you carefully design for it, and don't pass around objects and callbacks and stuff.

I have gone through several iterations of abstractions trying to make updatable objects and such to hide this from you, and none were very good.

I eventually settled on a message bus, and a data structure I'm calling a Tag Point adapted from the SCADA industry, which is like a variable, but you can subscribe to it, and spy on the value from the web UI, it stays around as long as there is a reference to it, and it's guaranteed that any tags with the same name are the same object, and a bunch of other stuff.

Files of code are essentially meant to be used like stateless microservices if you expect to update one, and if you want to access shared stuff, you make sure to not hold a reference to it.

A lot was(and some still is) based on weak references, and those can trouble and should probably not be relied on for correctness if possible.

They aren't the worst things though. I have code to run a few GC sweeps when a file is deleted, and it is extremely rare that anything stays around when it shouldn't.

Good enough for development, good enough for emergency fixes, not the best for regular updates to running systems, although it's 99.9% fine, and I can't say I really worry about it(But future versions should be more deterministic and more suited for live edits as a regular practice).

Erlang seems to like its functional-ness and I imagine that's a huge asset. Language level support is definitely a good thing.

Random unstructured code doesn't seem to work well with live updates.

If you are working in a general language like Python you really want to have your engine always know exactly what's going on, what subscriptions come from what module and which function is replacing what, etc.

You want to deterministically always be able to list any changes that a module made to anything else(Like subscribing to a function), and undo them.

But... weak refs work well enough.

The classic solution to tech problems is to reboot, live updates are kind of the opposite of that. Your new code has to perfectly pick up with what you keep from the old state, and if the old state is invalid somehow you have to deal with that too.

I've never heard of smalltalk as a live update language(In the Erlang telephone exchange sense), just that you can do interactive development in it, which is a lot easier.

I think if I intended to seriously to true live updates(Like phone exchanges not dropping active calls), I would really appreciate tools made for that.

Kaithem also does have a module with a visual script explicitly meant for changes in production. It's very limited and opinionated, essentially a state machine with event triggered actions attached to states, and variables at the state machine level.

That kind of transition rule system for simple tasks is very easy to live update. State is well separated from rules, and rules are simple and easy to parse programmatically, to do things like clean up after yourself if an event listener needs resources.

But just about any interpreted language works for interactive development.

A lot of people like FORTH for that, which I don't have any interest in learning but some love it.

Mine is Scheme for Max, now on it's fourth open source release, but really written so I could make computer music how I want to. It's an extension to the popular Max/MSP visual music programming environment that embeds an s7 Scheme interpreter and provides a substantial API/FFI to Max. It allows you to script Max (and thus also Ableton Live) with Scheme, enabling interactive coding, algorithmic music, live coding, macros, and just much more pleasant scripting than in JavaScript. It locks in with the scheduler so you can even use Scheme powered sequencers within Ableton Live alongside regular Live tracks, and you can build sophisticated Live control surfaces using the Live API.

Github page here: https://github.com/iainctduncan/scheme-for-max

The project page is here, with links to lots of documentation I've done: https://github.com/iainctduncan/scheme-for-max

To download Scheme for Max and for tutorials, documentation, and the cookbook, visit the GitHub page: https://github.com/iainctduncan/scheme-for-max

If you like things like Janet, you might also like s7 Scheme. It is also a minimal Scheme built entirely in C and dead easy to embed. I used it to make Scheme for Max and Scheme for Pd, extensions to the Max and Pd computer music platform to allow scripting them in Scheme. (https://github.com/iainctduncan/scheme-for-max) Janet was one of the options I looked pretty closely at before choosing s7.

The author (Bill Schottstaedt, Stanford CCRMA) is not too interested in making pretty web pages, ha, but the language is great!

Project page: https://github.com/iainctduncan/scheme-for-max

I created Scheme for Max and Scheme for Pure Data. They are extensions to the Max/MSP, Ableton Live, and Pure Data computer music environments that embed an s7 Scheme interpreter in the host so that you can script, automate, and live code the hosts with s7, a Scheme from the CCRMA computer music center at Stanford and the same one used in the Snd editor and the Common Music 3 algorithmic composition environment. This allows you to do things like write algorithmic music tools, sequencers, and use the Ableton Live API in Scheme, including with Common Lisp style macros. It has an API for integrating with Max to share data structures, hook into the scheduler, run in the high priority thread, and so on. S4M allows you to do all the goodness of high level music programming in a Lisp, without losing the ability to use modern commercial tooling and instruments. It's my thesis project for a Masters in Music Technology with Andy Schloss and George Tzanetakis at the University of Victoria, and I plan to continue to a PhD working on it. I tried submitting twice, but it never made the page, which surprised me a bit given Lisp interest here.

The github page is here: https://github.com/iainctduncan/scheme-for-max

The youtube channel with various demos is here: https://www.youtube.com/c/musicwithlisp