py.test and tox

It is easy to integrate py.test runs with tox. If you encounter issues, please check if they are listed as a known issue and/or use the support channels.

Basic example

Assuming the following layout:

tox.ini      # see below for content
setup.py     # a classic distutils/setuptools setup.py file

and the following tox.ini content:

[tox]
envlist = py26,py31

[testenv]
deps=pytest       # PYPI package providing py.test
commands=
  py.test \
        {posargs} # substitute with tox' positional arguments

you can now invoke tox in the directory where your tox.ini resides. tox will sdist-package your project, create two virtualenv environments with the python2.6 and python3.1 interpreters, respectively, and will then run the specified test command in each of them.

Extended example: change dir before test and use per-virtualenv tempdir

Assuming the following layout:

tox.ini      # see below for content
setup.py     # a classic distutils/setuptools setup.py file
tests        # the directory containing tests

and the following tox.ini content:

[tox]
envlist = py26,py31
[testenv]
changedir=tests
deps=pytest
commands=
  py.test \
    --basetemp={envtmpdir}  \ # py.test tempdir setting
    {posargs} # substitute with tox' positional arguments

you can invoke tox in the directory where your tox.ini resides. Differently than in the previous example the py.test command will be executed with a current working directory set to tests and the test run will use the per-virtualenv temporary directory.

Using multiple CPUs for test runs

py.test supports distributing tests to multiple processes and hosts through the pytest-xdist plugin. Here is an example configuration to make tox use this feature:

[testenv]
deps=pytest-xdist
changedir=tests
commands=
    py.test \
        --basetemp={envtmpdir}  \
        --confcutdir=..         \
        -n 3                    \ # use three sub processes
        {posargs}

Known Issues and limitations

Too long filenames. you may encounter “too long filenames” for temporarily created files in your py.test run. Try to not use the “–basetemp” parameter.

installed-versus-checkout version. py.test collects test modules on the filesystem and then tries to import them under their fully qualified name. This means that if your test files are importable from somewhere then your py.test invocation may end up importing the package from the checkout directory rather than the installed package.

There are a few ways to prevent this.

With installed tests (the tests packages are known to setup.py), a safe and explicit option is to give the explicit path {envsitepackagesdir}/mypkg to pytest. Alternatively, it is possible to use changedir so that checked-out files are outside the import path, then pass --pyargs mypkg to pytest.

With tests that won’t be installed, the simplest way to run them against your installed package is to avoid __init__.py files in test directories; pytest will still find and import them by adding their parent directory to sys.path but they won’t be copied to other places or be found by Python’s import system outside of pytest.