FsCheck VS pynguin

On the left corner, weighing in at v3.x, hailing from the OG PBT library, Haskell's QuickCheck, of which it's a direct port, we have FsCheck - F# premier PBT library, well established, battle-tested, community-driven, just solid.

As you can see from the imports statement we're relying on FsCheck to generate some random values for us.

C# - FsCheck

https://fscheck.github.io/FsCheck/ can hopefully generate random inputs automatically or with low effort for many methods to get your code coverage up. You don’t even need to write real tests right now, just call the methods with the random inputs and check they don’t fail.

Try using F# for tests. It has some great libraries like FsCheck (https://fscheck.github.io/FsCheck/).

At Symbolica we're building a symbolic execution service that explores every reachable state of a user's program and verifies assertions at each of these states to check that the program is correct. By default it will check for common undefined behaviours, such as out-of-bounds memory reads or divide by zero, but it can also be used with custom, application specific, assertions too just like the kind you'd write in a unit test. Seen from this perspective it's kind of like FsCheck (or Haskell's QuickCheck or Python's Hypothesis), but much more exhaustive and without the randomness.

Maybe something like QuickCheck, a quick search gave me this library for .NET https://github.com/fscheck/FsCheck

Because of the above I've generally been using tools like Stryker.NET and FsCheck to augment my testing suite. I'm still doing unit testing to find the more obvious "I haven't had my coffee, let's make sure I'm doing what I think I'm doing" bugs. I'm just using things like mutation testing, property testing, fuzzing, etc. to find the deeper issues in my code. There's a ton of libraries out there, including one that I've built for myself to help with testing but FsCheck and Stryker are just beautiful. And if you're interested in fuzzing, SharpFuzz is a great option. But that one isn't quite as easy of an on ramp compared to the other two that I mentioned.

Looks cool. Is there a reason why you didn't use FsCheck or Hedgehog? They're built to generate random data for testing, and can return the seed if a test fails so you can rerun the test with the exact same data once you figure out what the problem is - which is useful if the failure condition is rare.

Haskell has QuickCheck and Hedgehog, and dotnet has both as well. F# is favored, but there's C# interop.

https://swagger.io/specification/ https://github.com/se2p/pynguin

I would use black to reformat this, then, if you can't refactor/rewrite (which is a lot of work!) I would try automated test generation via something like pynguin or fuzzing. I mean … this is not going to be a reliable solution anyways if the codebase is like that. So I would go in a direction that I find interesting to learn about and that could be helpful for the project. That would be generating tests and doing fuzzing. In the end you should run some linters anyways so that you can justify your results and show that the task is not in the scope of an internship and needs extensive refactoring.

The main difference that Klara bring to the table, compared to similar tool like pynguin and Crosshair is that the analysis is entirely static, meaning that no user code will be executed, and you can easily extend the test generation strategy via plugin loading (e.g. the options arg to the Component object returned from function above is not needed for test coverage).

Automated White-box test generation software: * https://github.com/EMResearch/EvoMaster -- for integration tests. * https://github.com/se2p/pynguin, https://pynguin.readthedocs.io/en/latest/user/quickstart.html -- unit test generation for python

Pynguin – Generate Python unit tests automatically\ (60 comments)