Gathering statistics

As your tests are ran, you may want to gain some insight into what test cases are actually generated in order to verify your properties. Usually, however, there will be hundreds or thousands of them, so you certainly don’t want to wade through them all.

To consolidate this data into more useful information, pyqcy provides you with statistical functions: collect() and classify().

Warning

All statistical functions described below must be yield from within test properties to be recorded.

pyqcy.statistics.collect(value)

Collects test cases that share the same value (passed as argument) for statistical purposes.

Parameters:value – Value to collect. This can be any hashable, i.e. a value that could be a set element or dictionary key.

Typical usage of collect() is as follows:

@qc
def sort_works(
    l=list_(int, min_length=1, max_length=100)
):
    yield collect(len(l))
    assert list(sorted(l))[0] == min(l)

Checking the above property will produce output similar to this:

sort_works: passed 100 tests.
1.00%: 1
1.00%: 2
...
1.00%: 100
pyqcy.statistics.classify(condition, label)

Classifies test cases depending on whether they satisfy given condition.

If a test case meets the condition, it will be “stamped” with given label that will subsequently appear in statistical report displayed after a property has been tested.

Parameters:
  • condition – Condition that the test data should satisfy in order for the test case to be stamped with label.
  • label – A label to be associated with this test case if condition turns out to be true

Typical usage is as follows:

@qc
def sort_preserves_length(
    l=list_(int, min_length=1, max_length=100)
):
    yield classify(len(l) == 0, "empty list")
    yield classify(len(l) < 10, "short list")
    assert len(list(sorted(l))) == len(l)

Checking the above property will produce something like the following output:

sort_preserves_length: passed 100 tests.
1.00%: empty list, short list
9.00%: short list
90.00%: <rest>