Hi everyone!
Property-based testing is amazing: You specify what types of input values you expect and what kinds of properties are expected to hold true for all of those possible inputs, and the runtime is then able to automatically create and run thousands of test-cases for you. This often uncovers edge-cases that you might not have thought of if you were trying to come up with unit-test examples by hand.
What is even more cool is that when a test failure is encountered, the input can be shrunken back to the most simplest form that still results in an error, making the feedback very human-friendly.
Property-based testing can be considered a ‘pragmatic’ approach to ‘theorem-proving’. Mathematically proving that something holds for all cases is often very difficult, whilst stating that something should hold for all cases and then lobbing thousands of possible input sets at it is a lot easier to make. Of course this only allows to prove the presence (rather than the absence) of bugs, but because of the large number of input sets (and the fact that every time you run the test, yet more different input sets are tried), it converges to the same.
Besides working in a couple of language where property-testing is wide-spread I do quite a bit of consulting work in Ruby. Until now, Ruby did not have a mature library to do property-based testing.
So I wrote one 
!
PropCheck
It features:
- Generators for common datatypes.
- An easy DSL to define your own generators (by combining existing ones, or make completely custom ones).
- Shrinking to a minimal counter-example on failure.
Usage Example
Here we check if naive_average indeed always returns an integer for any and all arrays of integers we can pass it:
# Somewhere you have this function definition:
def naive_average(array)
array.sum / array.length
end
# And then in a test case:
include PropCheck::Generators
PropCheck.forall(numbers: array(integer)) do |numbers:|
result = naive_average(numbers)
unless result.is_a?(Integer) do
raise "Expected the average to always return an integer!"
end
end
When running this particular example PropCheck very quickly finds out that we have made a programming mistake:
ZeroDivisionError:
(after 6 successful property test runs)
Failed on:
`{
:numbers => []
}`
Exception message:
---
divided by 0
---
(shrinking impossible)
---
Clearly we forgot to handle the case of an empty array being passed to the function.
This is a good example of the kind of conceptual bugs that PropCheck (and property-based testing in general) are able to check for.
(If we were e.g. using RSpec, we might have structured the test as follows:
describe "#naive_average" do
include PropCheck
include PropCheck::Generators
it "returns an integer for any input" do
forall(numbers: array(integer)) do |numbers:|
result = naive_average(numbers)
expect(result).to be_a(Integer)
end
end
end
)
PropCheck comes with many built-in data-generators and it is easy to build your own on top of these.
Check it out now! I’m eager to hear your feedback !