Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
The approach of check-pointing computation such that it is resumable and restartable sounds similar to a time-traveling debugger, like rr or WinDbg:
https://rr-project.org/
https://learn.microsoft.com/windows-hardware/drivers/debugge...
Flawless sounds a lot like https://temporal.io/ .
I'm wondering if it has the same scalability concerns - sticking everything in Postgres is fine at small-ish scale, but what happens when you outgrow Postgres, either because you have higher availability requirements (can't handle primary DB restarts) or because of the sheer volume of the workload?
Indeed. I do wish there were a few more "extras" in OTP, because "let it crash" needs some more details in some circumstances.
For instance if you have a system with a user interface and some various components, like say, a database, and the database becomes unavailable, you don't want the entire system to crash. You want it to display an error message to the user and maybe go into some kind of diagnostic mode or other "things are not normal" state.
Something like https://github.com/jlouis/fuse is one approach.
Checkpoint/Restore I feel is a bigger concept than just saving state. At the zeroth level it's a system that can correctly stop and serialize a running process (as criu https://github.com/checkpoint-restore/criu has shown is a huge pain in the ass to still not be perfect) in a way that can initiated from within the process itself.
The 1st level more-work-but-easier way to do this is to build or use a heavily constrained VM/language you run from within your main application that doesn't allow for most of the hard problems to even exist.
I can't find any ready-made tools to do this that I wouldn't consider an endeavor.