telepythy VS outrun

Hmm interesting idea. I was working on something a couple weeks ago after all the "αcτµαlly pδrταblε εxεcµταblε" stuff. My idea was to be able to run local programs like your favorite fancy shell, but on a remote machine or container that does not have it installed (think lightweight containers you need to work in). The idea was to have a parent program that runs your shell or other client program using ptrace to intercept and proxy all syscalls to a small client on the remote machine/container. So the code would be running locally, but all of the syscalls would be running remotely. I actually got it somewhat working but gave up when I realized that the difficulty in memory and file access. Files in particular were hard since I couldn't disambiguate if a file access was for a "local" or "remote" file. Also in the past I did something silmilar for python programs https://github.com/seiferteric/telepythy

While it's purpose is different it can be used to do distributed compiling, so I'll leave it here.

https://github.com/Overv/outrun

Since I was just going down this rabbit hole recently, I kind of wonder if it's possible to set the filesystem on something more like the BitTorrent protocol so things like the libraries/compilers/headers that are used during compilation dont all need to come from the main pc. It probably wouldn't be useful until you reached a stupid number of computers and you started reaching the limits of the Ethernet wire, but for something stupid that can run on a pi cluster it would be a fun project.

Your mention of compile heavy workloads reminded me of a project called Outrun, it offloads work to another machine. All it seems to require is Python, Fuse3 and ssh.

Awesome! This write up is satisfyingly detailed. Prior work in this space includes Plan9 of course, as well as the python project Outrun, which has it's own RPC-based FUSE FS: https://github.com/Overv/outrun

Other approachs to deployment in particular include the functional package managers Nix and Guix, which can create lightweight application images, and could probably be cobbled together into some sort of remote environment replication even across architectures. As I read on, I thought less about how this compares with Guix in regards to application/environment packaging and more about how these things could be glued together in interesting ways, because I think the intro leads in through slightly off-label examples, if that makes sense. Application packaging isn't what this addresses at the end of the day, but it's no less fascinating for it.