Our great sponsors
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
hermit
Hermit launches linux x86_64 programs in a special, hermetically isolated sandbox to control their execution. Hermit translates normal, nondeterministic behavior, into deterministic, repeatable behavior. This can be used for various applications, including replay-debugging, reproducible artifacts, chaos mode concurrency testing and bug analysis. (by facebookexperimental)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
- Dropbox [3] uses a similar approach but they talk about it a bit more abstractly.
Sans-IO is more documented in Python [4], but str0m [5] and quinn-proto [6] are the best examples in Rust I’m aware of. Note that sans-IO is orthogonal to deterministic test frameworks, but it composes well with them.
With the disclaimer that my opinions are mine and mine alone, and don’t reflect the company I work at —— I do work at a rust shop that has utilized these techniques on some projects.
TigerBeetle is an amazing example and I’ve looked at it before! They are really the best example of this approach outside of FoundationDB I think.
[0]: https://risingwave.com/blog/deterministic-simulation-a-new-e...
[1]: https://risingwave.com/blog/applying-deterministic-simulatio...
[2]: https://dropbox.tech/infrastructure/-testing-our-new-sync-en...
[3]: https://github.com/spacejam/sled
[4]: https://fractalideas.com/blog/sans-io-when-rubber-meets-road...
[5]: https://github.com/algesten/str0m
[6]: https://docs.rs/quinn-proto/0.10.6/quinn_proto/struct.Connec...
- Dropbox [3] uses a similar approach but they talk about it a bit more abstractly.
Sans-IO is more documented in Python [4], but str0m [5] and quinn-proto [6] are the best examples in Rust I’m aware of. Note that sans-IO is orthogonal to deterministic test frameworks, but it composes well with them.
With the disclaimer that my opinions are mine and mine alone, and don’t reflect the company I work at —— I do work at a rust shop that has utilized these techniques on some projects.
TigerBeetle is an amazing example and I’ve looked at it before! They are really the best example of this approach outside of FoundationDB I think.
[0]: https://risingwave.com/blog/deterministic-simulation-a-new-e...
[1]: https://risingwave.com/blog/applying-deterministic-simulatio...
[2]: https://dropbox.tech/infrastructure/-testing-our-new-sync-en...
[3]: https://github.com/spacejam/sled
[4]: https://fractalideas.com/blog/sans-io-when-rubber-meets-road...
[5]: https://github.com/algesten/str0m
[6]: https://docs.rs/quinn-proto/0.10.6/quinn_proto/struct.Connec...
That'll work great for your Distributed QSort Incorporated startup, where the only product is a sorting algorithm.
Formal software verification is very useful. But what can be usefully formalized is rather limited, and what can be formalized correctly in practice is even more limited. That means you need to restrict your scope to something sane and useful. As a result, in the real world running thousands of tests is practically useful. (Well, it depends on what those tests are; it's easy to write 1000s of tests that either test the same thing, or only test the things that will pass and not the things that would fail.) They are especially useful if running in a mode where the unexpected happens often, as it sounds like this system can do. (It's reminiscent of rr's chaos mode -- https://rr-project.org/ linking to https://robert.ocallahan.org/2016/02/introducing-rr-chaos-mo... )
Exactly - that's what we've already built for web development at https://replay.io :)
I did a "Learn with Jason" show discussion that covered the concepts of Replay, how to use it, and how it works:
- https://www.learnwithjason.dev/travel-through-time-to-debug-...
Not only is the debugger itself time-traveling, but those time-travel capabilities are exposed by our backend API:
- https://static.replay.io/protocol/
Our entire debugging frontend is built on that API. We've also started to build new advanced features that leverage that API in unique ways, like our React and Redux DevTools integration and "Jump to Code" feature:
- https://blog.replay.io/how-we-rebuilt-react-devtools-with-re...
- https://blog.isquaredsoftware.com/2023/10/presentations-reac...
- https://github.com/Replayio/Protocol-Examples
Exactly - that's what we've already built for web development at https://replay.io :)
I did a "Learn with Jason" show discussion that covered the concepts of Replay, how to use it, and how it works:
- https://www.learnwithjason.dev/travel-through-time-to-debug-...
Not only is the debugger itself time-traveling, but those time-travel capabilities are exposed by our backend API:
- https://static.replay.io/protocol/
Our entire debugging frontend is built on that API. We've also started to build new advanced features that leverage that API in unique ways, like our React and Redux DevTools integration and "Jump to Code" feature:
- https://blog.replay.io/how-we-rebuilt-react-devtools-with-re...
- https://blog.isquaredsoftware.com/2023/10/presentations-reac...
- https://github.com/Replayio/Protocol-Examples
I really like antithesis' approach: it's non-intrusive as all the changes are on a VM so one can run deterministic simulation without changing their code. It's also technically challenging, as making a VM suitable for deterministic simulation is not an easy feat.
On a side, I was wondering how this approach compares to Meta's Hermit(https://github.com/facebookexperimental/hermit), which is a deterministic Linux instead of a VM.
The article says they created a deterministic hypervisor that runs all pseudorandom behavior from a starting seed to enable perfect re-playability.
But that's all we know so far. I'm assuming there'll be some sort of fuzz testing, and static analysis or some defining actions that your software can perform.
Honestly it sounds a lot like it has a lot of crossover with what the Vale language is trying to solve: https://vale.dev/, but focused on trying to get existing software to that state instead of creating a new language to make new software already be at that state by default.
Along similar lines, Mario Carneiro wrote a formalisation of a subset of x86 in MetaMath Zero (https://github.com/digama0/mm0/blob/master/examples/x86.mm0) with the ultimate goal of proving that the MetaMath Zero verifier itself is sound. https://arxiv.org/pdf/1910.10703.pdf
(And of course Permutation City is a fiction book all about emulating computers with sound properties!)