PropCheck - Ruby library for property-based testing

Hi everyone!

Property-based testing is amazing: You specify what types of input values you expect and what kinds of properties are expected to hold true for all of those possible inputs, and the runtime is then able to automatically create and run thousands of test-cases for you. This often uncovers edge-cases that you might not have thought of if you were trying to come up with unit-test examples by hand.

What is even more cool is that when a test failure is encountered, the input can be shrunken back to the most simplest form that still results in an error, making the feedback very human-friendly.

Property-based testing can be considered a ‘pragmatic’ approach to ‘theorem-proving’. Mathematically proving that something holds for all cases is often very difficult, whilst stating that something should hold for all cases and then lobbing thousands of possible input sets at it is a lot easier to make. Of course this only allows to prove the presence (rather than the absence) of bugs, but because of the large number of input sets (and the fact that every time you run the test, yet more different input sets are tried), it converges to the same.

Besides working in a couple of language where property-testing is wide-spread I do quite a bit of consulting work in Ruby. Until now, Ruby did not have a mature library to do property-based testing.
So I wrote one :smiley:!


Gem Build Status Maintainability RubyDoc

It features:

  • Generators for common datatypes.
  • An easy DSL to define your own generators (by combining existing ones, or make completely custom ones).
  • Shrinking to a minimal counter-example on failure.

Usage Example

Here we check if naive_average indeed always returns an integer for any and all arrays of integers we can pass it:

# Somewhere you have this function definition:
def naive_average(array)
  array.sum / array.length

# And then in a test case:
include PropCheck::Generators
PropCheck.forall(numbers: array(integer)) do |numbers:|
  result = naive_average(numbers)
  unless result.is_a?(Integer) do
    raise "Expected the average to always return an integer!"

When running this particular example PropCheck very quickly finds out that we have made a programming mistake:

(after 6 successful property test runs)
Failed on: 
    :numbers => []

Exception message:
divided by 0

(shrinking impossible)

Clearly we forgot to handle the case of an empty array being passed to the function.
This is a good example of the kind of conceptual bugs that PropCheck (and property-based testing in general) are able to check for.

(If we were e.g. using RSpec, we might have structured the test as follows:

describe "#naive_average" do
  include PropCheck
  include PropCheck::Generators

  it "returns an integer for any input" do
    forall(numbers: array(integer)) do |numbers:|
      result = naive_average(numbers)      
      expect(result).to be_a(Integer)


PropCheck comes with many built-in data-generators and it is easy to build your own on top of these.

Check it out now! I’m eager to hear your feedback :smiley:!


Nice one Marten!


Version 0.10 has been released! Besides a couple of bugfixes, the new version adds before/after/around callbacks that you can use to add setup and/or teardown logic that is run before/after/around every time a check with newly generated data is run.


Very cool @Qqwy!

It really is!

My first exposure was last summer when I read the following book; it was kinda mind blowing.


I very much recommend that book as well. Much of what I know about Property-based testing I learned from that book.

After finishing it I looked into the implementations of Elixir’s StreamData, Erlang’s PropEr and Haskell’s QuickCheck and Hedgehog to get some of the final details down that are necessary to actually implement your own property-checking library (like how shrinking actually works under the hood. It is amazing :exploding_head:).


Please consider writing reviews for books - they show up in the book’s portal and could really help others (as well as the authors) :blush: