jsverify VS fast-check

Libraries like JSVerify or Fast-Check offer essential tools to facilitate property-based testing.

I tend to use anything that offers property-testing, since tests are much shorter to write and uncover lots more hidden assumptions.

My go-to choices per language are:

- Python: Hypothesis https://hypothesis.readthedocs.io/en/latest (also compatible with PyTest)

- Scala: ScalaCheck https://scalacheck.org (also compatible with ScalaTest)

- Javascript/Typescript: JSVerify https://jsverify.github.io

- Haskell: LazySmallCheck2012 https://github.com/UoYCS-plasma/LazySmallCheck2012/blob/mast...

- When I wrote PHP (over a decade ago) there was no decent property-based test framework, so I cobbled one together https://github.com/Warbo/php-easycheck

All of the above use the same basic setup: tests can make universally-quantified statements (e.g. "for all (x: Int), foo(x) == foo(foo(x))"), then the framework checks that statement for a bunch of different inputs.

Most property-checking frameworks generate data randomly (with more or less sophistication). The Haskell ecosystem is more interesting:

- QuickCheck was one of the first property-testing frameworks, using random genrators.

- SmallCheck came later, which enumerates data instead (e.g. testing a Float might use 0, 1, -1, 2, -2, 0.5, -0.5, etc.). That's cute, but QuickCheck tends to exercise more code paths with each input.

- LazySmallCheck builds up test data on-demand, using Haskell's pervasive laziness. Tests are run with an error as input: if they pass, we're done; if they fail, we're done; if they trigger the error, they're run again with slightly more-defined inputs. For example, if the input is supposed to be a list, we try again with the two forms of list: empty and "cons" (the arguments to cons are both errors, to begin with). This exercises even more code paths for each input.

- LazySmallCheck2012 is a more versatile "update" to LazySmallCheck; in particular, it's able to generate functions.

The usage of hypothesis is very intuitive and simple, and presents the concept of property-based testing perfectly. So I also wanted to find an equivalent alternative in Node. Two of them have high star ratings on Github, JSVerify with 1.6K stars and fast-check with 2.8K stars. So I took some time to study fast-check a little bit and try to get closer to my daily work.

Systems I've used for this include https://agda.readthedocs.io/en/v2.6.0.1/getting-started/what... https://coq.inria.fr https://www.idris-lang.org and https://isabelle.in.tum.de

An easier alternative is to try disproving the statement, by executing it on thousands of examples and seeing if any fail. That gives us less confidence than a full proof, but can still be better than traditional "there exists" tests. This is called property checking or property-based testing. Systems I've used for this include https://hypothesis.works https://hackage.haskell.org/package/QuickCheck https://scalacheck.org and https://jsverify.github.io

Using create-react-app, you’ll run npm test which uses Jest internally. If you are dealing with a lot of data on the UI, or using TypeScript, use JSVerify for property tests. For end to end tests, Cypress is a great choice.

Libraries like JSVerify or Fast-Check offer essential tools to facilitate property-based testing.

Strong agree!

For JavaScript, I suggest folks check out fast-check [0] and this introduction to property-based testing that uses fast-check [1].

This is broadly useful, but one specific place I've found it helpful was to check redux reducers against generated lists of actions to find unchecked edge cases and data assumptions.

[0] https://github.com/dubzzz/fast-check

Brilliant, this is a massive improvement. Previously I was fumbling around in https://github.com/dubzzz/fast-check/tree/main/packages/fast-check/documentation for info.

There's a library for exactly that: FastCheck.

It makes @fast-check/jest, the best option to integrate Jest and fast-check, as it provides an abstraction over both to ease their mutual integration.

The closest that I know of (and I have not used this) is zod-fast-check. It generates fast-check “arbitraries” (test data generators) for property-based testing based on zod schemas. Of course, this requires that you use zod to define your types, which has some downsides. Fortunately there is another tool, ts-to-zod, (which I also have not used) which will codegen zod schemas based on TS type definitions. If you thread these four libraries together you should end up with the ability to write random tests on generated data with very little overhead. In theory.

For something like this, I might break out fast-check for testing. It’s good at generating a wide range of values for a given type, and could help you get good test coverage without having to hand-author a lot of repetitive error inputs.