KaithemAutomation VS ipyflow

My SD protection script, a few months old and may need some updates since a lot seems to have changed in Pi OS:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Current version doesn't disable swap, that's in a separate optional file, but the next version will.

It can't be done in a one size fits all script unless you're launching chromium the same way, but do something like:

I don't usually do full read only, what I'll do is run a script that turns off stuff that does not need to be writing to the disk all the time.

Unfortunately, some software is database-oriented and likes to write to disk for every tiny thing, so the approach doesn't work with stuff like Home Assistant unless you carefully configure logging.

The basic simple stuff doesn't really cause any user-level noticable changes:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

After that, I disable and mask apt-daily (The Debian auto updater), and purge dphys-swapfile.

My full set of assorted tweaks can be found here, some might not be relevant for you:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Next, I often run Chromium as a kiosk, and Chromium likes to hammer the SD card, so I set the XDG folder environment variables to make it put it's stuff in RAM. My embedded chrome stuff can be found here:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Things like that are easy enough to set up on Raspbian with a script. You also need a bunch of tmpfs mounts, including some in crazy places(Regular Pi has a log file under ~/.cache that can fill disks and crash servers, and if you don't fix it Chromium writes seemingly useless crap constantly. There's auto updates on some systems, which is terrible if you're on a private WiFi, not doing internet stuff.

I'm not using zram at the moment, just getting rid of swap, but my current script to get a Pi ready for embedded projects is here: https://github.com/EternityForest/KaithemAutomation/blob/mas...

The only FOSS thing I've done that I think is really worth telling people about is KaithemAutomation, a home automation server in pure Python with a bit easier setup than Home Assistant, and some features aimed at commercial installs like room escape control, and some pretty decent network video recorder features.

https://github.com/EternityForest/KaithemAutomation

I put 6 years or so into it, and have used it on plenty of contract projects, but so far I don't think anyone else is interested.

Possibly because it's largely UI and CRUD over existing functionality, and there's not much particularly exciting to the hacker community, few interesting algorithms, it's not minimalist at all, etc.

Plus it has a lot of dependencies that might or might not exist outside of Debian, I've never looked into how it would run on the more DIY distros since I've never used them.

Then after that I square every pixel and square root the mean of the whole image.

I forgot how complicated this was and how many tweaks I added!

Code here: https://github.com/EternityForest/KaithemAutomation/blob/mas...

And for the motion detector specifically:

https://github.com/EternityForest/KaithemAutomation/blob/9db...

pure Python and heavily built around runtime modification(https://github.com/EternityForest/KaithemAutomation), and is designed to run for months, although it's not meant to be modified literally while running production(Not that that stops us from making last minute fixes...).

The big problem with it in Python is state. If you have a reference to something, and that something changes, and you have a reference to the old one, that should be a singleton, you are probably in an unhappy position.

The only way it can work in real life(In Python at least) is if you carefully design for it, and don't pass around objects and callbacks and stuff.

I have gone through several iterations of abstractions trying to make updatable objects and such to hide this from you, and none were very good.

I eventually settled on a message bus, and a data structure I'm calling a Tag Point adapted from the SCADA industry, which is like a variable, but you can subscribe to it, and spy on the value from the web UI, it stays around as long as there is a reference to it, and it's guaranteed that any tags with the same name are the same object, and a bunch of other stuff.

Files of code are essentially meant to be used like stateless microservices if you expect to update one, and if you want to access shared stuff, you make sure to not hold a reference to it.

A lot was(and some still is) based on weak references, and those can trouble and should probably not be relied on for correctness if possible.

They aren't the worst things though. I have code to run a few GC sweeps when a file is deleted, and it is extremely rare that anything stays around when it shouldn't.

Good enough for development, good enough for emergency fixes, not the best for regular updates to running systems, although it's 99.9% fine, and I can't say I really worry about it(But future versions should be more deterministic and more suited for live edits as a regular practice).

Erlang seems to like its functional-ness and I imagine that's a huge asset. Language level support is definitely a good thing.

Random unstructured code doesn't seem to work well with live updates.

If you are working in a general language like Python you really want to have your engine always know exactly what's going on, what subscriptions come from what module and which function is replacing what, etc.

You want to deterministically always be able to list any changes that a module made to anything else(Like subscribing to a function), and undo them.

But... weak refs work well enough.

The classic solution to tech problems is to reboot, live updates are kind of the opposite of that. Your new code has to perfectly pick up with what you keep from the old state, and if the old state is invalid somehow you have to deal with that too.

I've never heard of smalltalk as a live update language(In the Erlang telephone exchange sense), just that you can do interactive development in it, which is a lot easier.

I think if I intended to seriously to true live updates(Like phone exchanges not dropping active calls), I would really appreciate tools made for that.

Kaithem also does have a module with a visual script explicitly meant for changes in production. It's very limited and opinionated, essentially a state machine with event triggered actions attached to states, and variables at the state machine level.

That kind of transition rule system for simple tasks is very easy to live update. State is well separated from rules, and rules are simple and easy to parse programmatically, to do things like clean up after yourself if an event listener needs resources.

But just about any interpreted language works for interactive development.

A lot of people like FORTH for that, which I don't have any interest in learning but some love it.

You're probably referring to nbgather (https://github.com/microsoft/gather), which shipped with VSCode for a while.

nbgather used static slicing to get all the code necessary to reconstruct some cell. I actually worked with Andrew Head (original nbgather author) and Shreya Shankar to implement something similar in ipyflow (but with dynamic slicing and a not-as-nice interface): https://github.com/ipyflow/ipyflow?tab=readme-ov-file#state-...

I have no doubt something like this will make its way into marimo's roadmap at some point :)

Np.

From https://news.ycombinator.com/context?id=35887168 re: ipyflow I learned about ReactiveX for Python (RxPY) https://rxpy.readthedocs.io/en/latest/ .

https://github.com/ipyflow/ipyflow :

> IPyflow is a next-generation Python kernel for Jupyter and other notebook interfaces that tracks dataflow relationships between symbols and cells during a given interactive session, thereby making it easier to reason about notebook state.

FWIU e.g. panda3d does not have a react or rxpy-like API, but probably does have a component tree model?

https://news.ycombinator.com/item?id=38527552 :

>> It actually looks like pygame-web (pygbag) supports panda3d and harfang in WASM

> Harfang and panda3d do 3D with WebGL, but FWIU not yet agents in SSBO/VBO/GPUBuffer

> Another giveaway is the ratio of stars to watchers / forks. I remember one project with thousands of stars but only 10 users "watching" it. They went on to raise a sizable seed round too.

Not necessarily indicative of foul play. I have two projects like this (https://github.com/smacke/ffsubsync and https://github.com/ipyflow/ipyflow) and I attribute it to not having great developer documentation.

It's not in the highlights, but one of the things that excites me most is this: https://docs.python.org/dev/whatsnew/3.12.html#pep-669-low-i...

> PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details.

Low-overhead instrumentation opens up a whole bunch of interesting interactive use cases (i.e. Jupyter etc.), and as the author of one library that relies heavily on instrumentation (https://github.com/ipyflow/ipyflow), I'm very keen to explore the possibilities here.

You’re looking for reactive kernels: https://github.com/ipyflow/ipyflow