KaithemAutomation VS osxphotos

My SD protection script, a few months old and may need some updates since a lot seems to have changed in Pi OS:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Current version doesn't disable swap, that's in a separate optional file, but the next version will.

It can't be done in a one size fits all script unless you're launching chromium the same way, but do something like:

I don't usually do full read only, what I'll do is run a script that turns off stuff that does not need to be writing to the disk all the time.

Unfortunately, some software is database-oriented and likes to write to disk for every tiny thing, so the approach doesn't work with stuff like Home Assistant unless you carefully configure logging.

The basic simple stuff doesn't really cause any user-level noticable changes:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

After that, I disable and mask apt-daily (The Debian auto updater), and purge dphys-swapfile.

My full set of assorted tweaks can be found here, some might not be relevant for you:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Next, I often run Chromium as a kiosk, and Chromium likes to hammer the SD card, so I set the XDG folder environment variables to make it put it's stuff in RAM. My embedded chrome stuff can be found here:

https://github.com/EternityForest/KaithemAutomation/blob/mas...

Things like that are easy enough to set up on Raspbian with a script. You also need a bunch of tmpfs mounts, including some in crazy places(Regular Pi has a log file under ~/.cache that can fill disks and crash servers, and if you don't fix it Chromium writes seemingly useless crap constantly. There's auto updates on some systems, which is terrible if you're on a private WiFi, not doing internet stuff.

I'm not using zram at the moment, just getting rid of swap, but my current script to get a Pi ready for embedded projects is here: https://github.com/EternityForest/KaithemAutomation/blob/mas...

The only FOSS thing I've done that I think is really worth telling people about is KaithemAutomation, a home automation server in pure Python with a bit easier setup than Home Assistant, and some features aimed at commercial installs like room escape control, and some pretty decent network video recorder features.

https://github.com/EternityForest/KaithemAutomation

I put 6 years or so into it, and have used it on plenty of contract projects, but so far I don't think anyone else is interested.

Possibly because it's largely UI and CRUD over existing functionality, and there's not much particularly exciting to the hacker community, few interesting algorithms, it's not minimalist at all, etc.

Plus it has a lot of dependencies that might or might not exist outside of Debian, I've never looked into how it would run on the more DIY distros since I've never used them.

Then after that I square every pixel and square root the mean of the whole image.

I forgot how complicated this was and how many tweaks I added!

Code here: https://github.com/EternityForest/KaithemAutomation/blob/mas...

And for the motion detector specifically:

https://github.com/EternityForest/KaithemAutomation/blob/9db...

pure Python and heavily built around runtime modification(https://github.com/EternityForest/KaithemAutomation), and is designed to run for months, although it's not meant to be modified literally while running production(Not that that stops us from making last minute fixes...).

The big problem with it in Python is state. If you have a reference to something, and that something changes, and you have a reference to the old one, that should be a singleton, you are probably in an unhappy position.

The only way it can work in real life(In Python at least) is if you carefully design for it, and don't pass around objects and callbacks and stuff.

I have gone through several iterations of abstractions trying to make updatable objects and such to hide this from you, and none were very good.

I eventually settled on a message bus, and a data structure I'm calling a Tag Point adapted from the SCADA industry, which is like a variable, but you can subscribe to it, and spy on the value from the web UI, it stays around as long as there is a reference to it, and it's guaranteed that any tags with the same name are the same object, and a bunch of other stuff.

Files of code are essentially meant to be used like stateless microservices if you expect to update one, and if you want to access shared stuff, you make sure to not hold a reference to it.

A lot was(and some still is) based on weak references, and those can trouble and should probably not be relied on for correctness if possible.

They aren't the worst things though. I have code to run a few GC sweeps when a file is deleted, and it is extremely rare that anything stays around when it shouldn't.

Good enough for development, good enough for emergency fixes, not the best for regular updates to running systems, although it's 99.9% fine, and I can't say I really worry about it(But future versions should be more deterministic and more suited for live edits as a regular practice).

Erlang seems to like its functional-ness and I imagine that's a huge asset. Language level support is definitely a good thing.

Random unstructured code doesn't seem to work well with live updates.

If you are working in a general language like Python you really want to have your engine always know exactly what's going on, what subscriptions come from what module and which function is replacing what, etc.

You want to deterministically always be able to list any changes that a module made to anything else(Like subscribing to a function), and undo them.

But... weak refs work well enough.

The classic solution to tech problems is to reboot, live updates are kind of the opposite of that. Your new code has to perfectly pick up with what you keep from the old state, and if the old state is invalid somehow you have to deal with that too.

I've never heard of smalltalk as a live update language(In the Erlang telephone exchange sense), just that you can do interactive development in it, which is a lot easier.

I think if I intended to seriously to true live updates(Like phone exchanges not dropping active calls), I would really appreciate tools made for that.

Kaithem also does have a module with a visual script explicitly meant for changes in production. It's very limited and opinionated, essentially a state machine with event triggered actions attached to states, and variables at the state machine level.

That kind of transition rule system for simple tasks is very easy to live update. State is well separated from rules, and rules are simple and easy to parse programmatically, to do things like clean up after yourself if an event listener needs resources.

But just about any interpreted language works for interactive development.

A lot of people like FORTH for that, which I don't have any interest in learning but some love it.

> Any method that I've found to clean them up (exporting the originals, deleting them from the library, and then re-importing the JPEGs only seems easiest) will lose all of the years of metadata that I've built up in the library.

The open source tool osxphotos (https://github.com/RhetTbull/osxphotos) can help with this. You can export the JPEG images while preserving metadata using the thrid-party exiftool utility:

`osxphotos export /path/to/export --has-raw --skip-raw --exiftool`

This exports all images that have a raw pair but skips the raw component then uses exiftool (https://exiftool.org/) to write the metadata (keywords, etc.) to the exported JPEG files. You can then re-import these into photos either by dragging them or by running `osxphotos import /path/to/export/*`

Both the export and import commands have many other options for controlling export directory, etc. `osxphotos help export` or `osxphotos docs` to open docs in browser. (Disclaimer: I'm the author of osxphotos)

See the issue tracker if you want to follow along. Hopefully this is an easy fix and I can push a patch today.

In response to a question on the osxphotos GitHub Discussions page, I wrote a quick script to do prune empty albums and folders from Photos that can be run with osxphotos (version 0.65.0 and later). You can run the script directly from GitHub without downloading it first via:

You could try opening the library with PowerPhotos, a commercial app that can manage multiple Photos libraries, to see if it can read it. You could also try my free open source command line tool, osxphotos. Install it then run this command in the Terminal: osxphotos info --library /path/to/the/library This should print out a list of information about the library: number of photos, number of albums, keywords in the library, etc. If that works, then osxphotos can read the library and can likely export the photos for you so you could re-import into a new library.

I'm the author of the osxphotos[0] tool mentioned in the article. For photos in an Apple Photos library, osxphotos gives you access to a rich set of metadata beyond what's in the actual EXIF/IPTC/XMP of the image. Apple performs object classification and other AI techniques on your images but generally doesn't expose this to the user. For example, photos are categorized as to object in them (dog, cat, breed of dog, etc.), rich reverse geolocation info (neighborhood, landmarks, etc.) and an interesting set of scores such as "overall aesthetic", "pleasant camera tilt", "harmonious colors", etc. These can be queried using osxphotos, either from the command line, or in your own python code. (Ref API docs[1])

For example, to find your "best" photos based on overall aesthetic score and add them to the album "Best Photos" you could run:

osxphotos query --query-eval "photo.score.overall > 0.8" --add-to-album "Best Photos"

To find good photos with trees in them you could try something like:

osxphotos query --query-eval "photo.score.overall > 0.5" --label Tree --add-to-album "Good Tree Photos"

There's quite a bit of other interesting data in Photos that you can explore with osxphotos. Run `osxphotos inspect` and it will show you all the metadata for whichever photo is currently selected in the Photos app.

[0] https://github.com/RhetTbull/osxphotos

osxphotos is my own tool for power users to interact with Photos from the command line: export, batch edit, sync metadata, import, etc.

osxphotos v0.60.8 adds initial alpha support for macOS Sonoma (macOS 14.0.0 / Photos 9.0). Everything seems to be working but if you are beta testing Sonoma and use osxphotos I'd welcome any feedback you have!

I'm the author of the free/open source tool osxphotos which provides several utilities fo working with Photos and exporting your photos. You can use the batch-edit feature to automatically add the album name as a keyword and I believe keywords are shared across users. (I don't use shared libraries so can't confirm this). I am working on a feature to then automatically re-create the albums from the keywords on the target library. For now the keywords is a partial work around.

If you want to edit batch metadata of photos that are in the Apple Photos app on a Mac, I'm the author of a free tool, osxphotos that includes a batch-edit command that will edit the metadata in the Photos library.