Tutorial

Start with writing test cases for our program

def foo(x):
    return x * x

in a file (let’s call it reg_test.py)

class RegressiveTest(RegressionTestCase):
    """our regression test case"""

    def test_almost_regressive_equal(self):
        self.assertAlmostRegressiveEqual(foo(1.01))
        self.assertAlmostRegressiveEqual(foo(1.11))

    def test_regressive_equal(self):
        self.assertAlmostRegressiveEqual(foo.__name__)
        self.assertAlmostRegressiveEqual(foo(2) > 0)

At first test run

$ python -m untittest reg_test.py

the return values are stored in files (more precise the argument values of assertRegressiveEqual and assertAlmostRegressiveEqual):

test/data/RegressiveTest/test_regressive_equal.json.zip
test/data/RegressiveTest/test_almost_regressive_equal.json.zip

Re-running

$ python -m untittest reg_test.py

will now use those data. If any values have changed AssertError will be raised as usual.

If the testcase may have changed (less or resp. more calls of assertRegressiveEqual and assertAlmostRegressiveEqual) some reference data will be left over or resp. missing.

So a LeftoverAssertValueError or resp. MissingAssertValueError will be raised.

Note: All file input/outout is done by the setUp() and tearDown() methods of the standard unittest framework. So on overwrite, don’t forget to call either super or enhance

def setUp(self):
    self.readResults()

def tearDown(self):
    self.validateResults()
    self.writeResults()

Hint: To avoid compression by zip archives set the class property compression of the RegressionTestCase class to False

RegressiveTest.compression=False

Hence

test/data/RegressiveTest/test_regressive_equal.json
test/data/RegressiveTest/test_almost_regressive_equal.json