The happy state of property-based testing in C#

I recently spotted Stevan's very thorough blog post about the sad state of property-based testing libraries on Hacker News. I emailed to make him aware of CsCheck which has both stateful and parallel testing, and an innovative approach to shrinking. In reply he encouraged me to write something about these improvements, hopefully to do some bridge building between the communities, as there are too many silos of information.

Random shrinking and other innovations

Most property-based testing libraries are a port of QuickCheck and share its tree-based shrinking. This is where a lazy tree of possible simpler samples is composed and tested. CsCheck differs in that generation and shrinking are both based on random samples. Each generated sample comes with a Size proxy which can be used to compare for shrinking. Only samples with a smaller Size are tested during shrinking.

This gives the following advantages over tree based shrinking libraries:

A more detailed comparison can be found here.

Other innovative functionality in CsCheck:

C# community pitch

It's not so easy in the C# community to get blog posts or new OSS libraries noticed. There doesn't seem to be anywhere you can submit them. I don't think there has been much of a take up of property-based testing in the community outside of some people who have used F# before.

A good example of the kind of limitation this creates is the ability to design a simple allocation algorithm of an integer over a list of weights. The algorithm seems like it should be easy (it's not), but everywhere I've seen it done it's not been correct, from multiple finance companies, twitter, excel and stackoverflow. The only correct algorithms I've seen coded are mine (1,2,3) found by using CsCheck.

Other blog examples of CsCheck use in optimising numeric algorithms and writing a high performance SIEVE LRU cache.

Some places to use random testing:

Why use it is discussed further here.