In praise of property-based testing (2019)

• fast-check

  3 4,090 9.8 TypeScript

  Property based testing framework for JavaScript (like QuickCheck) written in TypeScript

  I've been aware of property-based testing for a number of years now, but never had a good opportunity to give it a try. Then the past year I had a piece of serialisation/de-serialisation code, which was the perfect opportunity for a rather simple property-based test. That gave me the hang of it, and found two (minor, but still) bugs.

  Then recently I had a fairly larger, more error prone piece of work that lend itself very well to property-based testing, and it's been a godsend. It helped me discover a number of bugs, this time with the risk of causing privilege escalation. And since the proptests started succeeding reliably, I've been very confident that a rather complex piece of code now actually does what it's supposed to.

  If you're working in JavaScript, I can recommend fast-check [1].

  Another interesting approach, that I haven't yet tried, is Quickstrom [2], basically Puppeteer for proptests. It opens a webpage in a browser, performs some random interactions (pressing buttons, entering data, etc.), and then verifies that properties you specified still hold.

  [1] https://dubzzz.github.io/fast-check.github.com/

  [2] https://quickstrom.io/

• proptest

  4 1,569 8.3 Rust

  Hypothesis-like property testing for Rust

  

  I'm quite new to property testing, first introduced recently via a Rust property testing framework proptest[0]. So far i've had the feeling that property testing frameworks need to include a way to rationalize complexity, as their assumptions can easily fall short as you illustrated.

  Eg the simplest example might be an application where you input an integer, but a smaller int actually drives up the complexity. This idea gets more complex when we consider a list of ints, where a larger list and larger numbers are simpler. Etcetc.

  It would be neat if a proptest framework supported (and maybe they do, again, i'm a novice here) a way to rank complexity. Eg a simple function which gives two input failures and you can choose which is the simpler.

  Another really neat way to do that might be to actually compute the path complexity as the program runs based on the count of CPU instructions or something. This wouldn't always be a parallel, but would often be the right default i imagine.

  Either way in my limited proptest experience, i've found it a bit difficult to design the tests to test what you actually want, but very helpful once you establish that.

  [0]: https://crates.io/crates/proptest

• InfluxDB

  www.influxdata.com

  sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project