Symbolica VS FsCheck

At Symbolica we're building a symbolic execution service that explores every reachable state of a user's program and verifies assertions at each of these states to check that the program is correct. By default it will check for common undefined behaviours, such as out-of-bounds memory reads or divide by zero, but it can also be used with custom, application specific, assertions too just like the kind you'd write in a unit test. Seen from this perspective it's kind of like FsCheck (or Haskell's QuickCheck or Python's Hypothesis), but much more exhaustive and without the randomness.

At Symbolica we’re building a cloud-hosted symbolic execution service. Symbolic execution lets you explore every reachable state of your program so that you can write tests without worrying about missing any edge cases. As a bonus we also automatically detect if any states can cause invalid memory access and other undefined behaviours, like divide by zero, without you having to write any additional tests.

View on GitHub

As you can see from the imports statement we're relying on FsCheck to generate some random values for us.

C# - FsCheck

https://fscheck.github.io/FsCheck/ can hopefully generate random inputs automatically or with low effort for many methods to get your code coverage up. You don’t even need to write real tests right now, just call the methods with the random inputs and check they don’t fail.

Try using F# for tests. It has some great libraries like FsCheck (https://fscheck.github.io/FsCheck/).

At Symbolica we're building a symbolic execution service that explores every reachable state of a user's program and verifies assertions at each of these states to check that the program is correct. By default it will check for common undefined behaviours, such as out-of-bounds memory reads or divide by zero, but it can also be used with custom, application specific, assertions too just like the kind you'd write in a unit test. Seen from this perspective it's kind of like FsCheck (or Haskell's QuickCheck or Python's Hypothesis), but much more exhaustive and without the randomness.

Maybe something like QuickCheck, a quick search gave me this library for .NET https://github.com/fscheck/FsCheck

Because of the above I've generally been using tools like Stryker.NET and FsCheck to augment my testing suite. I'm still doing unit testing to find the more obvious "I haven't had my coffee, let's make sure I'm doing what I think I'm doing" bugs. I'm just using things like mutation testing, property testing, fuzzing, etc. to find the deeper issues in my code. There's a ton of libraries out there, including one that I've built for myself to help with testing but FsCheck and Stryker are just beautiful. And if you're interested in fuzzing, SharpFuzz is a great option. But that one isn't quite as easy of an on ramp compared to the other two that I mentioned.

Looks cool. Is there a reason why you didn't use FsCheck or Hedgehog? They're built to generate random data for testing, and can return the seed if a test fails so you can rerun the test with the exact same data once you figure out what the problem is - which is useful if the failure condition is rare.

Haskell has QuickCheck and Hedgehog, and dotnet has both as well. F# is favored, but there's C# interop.

Side note, if you are thinking about testing in general, might be interested in property based testing. See for example https://fscheck.github.io/FsCheck/