Our great sponsors
-
MarkovJunior
Probabilistic language based on pattern matching and constraint propagation, 153 examples
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
jetscii
A tiny library to efficiently search strings for sets of ASCII characters and byte slices for sets of bytes.
I've done a similar thing myself when trying to test operations on a hierarchical database for my internal product.
Biggest difficulty for me was that some combinations of operations were illegal and I had to think about all the edge cases and filter them out, and this took a long time.
My SO works in hardware, and she says they always use a constraint solver to generate test cases / test vectors, and she never understood why this wasn't popular in software. I googled it a bit and found lots of academic papers but no concrete implementation.
I've also thought about generating test vectors using a generator based on Markov chains, I wrote about that here [0], based on this [1] submission.
I'm not familiar enough with either using constraint solvers to generate test cases, or Markov chains, to know if I'm talking nonsense here or is it just something that nobody thought to develop properly.
[0] https://news.ycombinator.com/item?id=31704791
[1] https://github.com/mxgmn/MarkovJunior
One also may bias values to be similar (or equal) to other recently generated values. This was a trick used in the compiler tester yarpgen, where it was useful in testing optimization of bitwise operators on integers.
https://github.com/intel/yarpgen
https://github.com/intel/yarpgen/blob/main/papers/yarpgen-oo...
(see "Policies for constants", page 196:8.)
If you enjoy reading Rust code, you can see some examples I have...
1. This example [1] compares a SIMD-accelerated implementation of an algorithm vs the naive implementation. This is usually referred to as an "oracle".
2. This example [2] tests an XML document object model library. The tests construct a sequence of operations to apply to a document ("add an element", "delete an element", "move an element" etc) and then assert properties that you expect to be true for a DOM tree (a parent and child are always cross-linked, for example)
[1]: https://github.com/shepmaster/jetscii/blob/8d7e44ad7da990ef1...
[2]: https://github.com/shepmaster/sxd/pull/21/files#diff-fc21cbf...
If you enjoy reading Rust code, you can see some examples I have...
1. This example [1] compares a SIMD-accelerated implementation of an algorithm vs the naive implementation. This is usually referred to as an "oracle".
2. This example [2] tests an XML document object model library. The tests construct a sequence of operations to apply to a document ("add an element", "delete an element", "move an element" etc) and then assert properties that you expect to be true for a DOM tree (a parent and child are always cross-linked, for example)
[1]: https://github.com/shepmaster/jetscii/blob/8d7e44ad7da990ef1...
[2]: https://github.com/shepmaster/sxd/pull/21/files#diff-fc21cbf...