OpenStack logo


Hack OpenStack



See also

nose usage and pbr (github).

testr: isolate bug

Load a subunit file downloaded from OpenStack gates:

(py27) haypo@smithers$ wget

(py27) haypo@smithers$ gunzip testrepository.subunit.gz (py27) haypo@smithers$ testr load testrepository.subunit


Run tests:

  • run –no-parallel: run all tests in a single process
  • testr run –until-failure: run forever, until a test fails

Analyze latest run:

  • testr last –subunit|subunit2ls: list tests of the previous run
  • testr run –analyze-isolation: try to isolate the failing test, find the minimum tests to reproduce the fail

subunit tools:

  • subunit-filter
  • subunit-ls

Bisection: diagnosing flaky tests.


Run unit tests:

. .tox/py27/bin/activate
testr run

Shell commands to run unit tests:

set -e && \
        TEMP_REZ=`mktemp -t` && \
        python testr --slowest --testr-args='--subunit  ' \
                | tee $$TEMP_REZ | subunit2pyunit || true ; \
        cat $$TEMP_REZ | subunit-filter -s --no-passthrough | subunit-stats ; \
        rm -f $$TEMP_REZ ;
  • --slowest shows the statistics at the end of the test run. Nothing fancy.
  • --testr-args='--subunit tells testr to output a subunit2 format for its unit tests. subunit2 format is a BINARY format, which you shouldn’t output to the screen.
  • subunit2pyunit will convert that to a nicer output
  • tee $$TEMP_REZ .. cat $$TEMP_REZ | subunit-filter -s --no-passthrough | subunit-stats shows the nice statistics about the test run (eg: how many tests in total, how many skips, how many failed, how many success)

testr: list skipped tests

testr last --subunit|subunit-filter -s|subunit-ls >A
testr last --subunit|subunit-filter -s --no-skip|subunit-ls >B
diff -u A B

tox/testr: “db type could not be determined” error

testr uses a database to store test results. If the database is created by Python 2, Python 3 cannot read it and then you get the error “db type could not be determined”.

Workaround: remove .testrepository directory and rerun tox again.

tox/testr: “gdbm is missing”

If you run tox -e py34 and then tox -e py27, the second commands may fail because Python 2.7 does not have the gdbm module.

On Ubuntu, type:

sudo apt-get install -y python-gdbm

testr: “local variable ‘run_subunit_content’ referenced before assignment” error

See Error message opaque when .testrepository files are unreadable.

Re-run a single failing test


Re-run a single test with testtools:

$ tox -e py33
FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
$ . .tox/py33/bin/activate
$ python -m tests.test_swiftclient.TestPutObject.test_unicode_ok
 Tests running...
 FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
 Ran 1 test in 0.002s


Re-run a single test with tox+testr:

$ tox -e py33
FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
$ tox -e py33 -- --isolated tests.test_swiftclient.TestPutObject.test_unicode_ok
FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok


Enter the virtualenv and type testr run tests.test_swiftclient.TestPutObject.test_unicode_ok should work, but it doesn’t in the Python 3.3 virtual environment of python-sphinxclient?!


Re-run a single test with nose:

$ nosetests
FAIL: tests.test_command_helpers.TestStatHelpers.test_stat_account_human

$ nosetests tests.test_command_helpers:TestStatHelpers.test_stat_account_human