Testing in Python

Defining test cases with unittest

Tests cases should be defined with the built-in unittest module. This allows for tests to be easily structured, and related tests can be grouped together.

Test assertions should be made with the assert statement, not the assertFoo methods provided by unittest; it's cleaner and easier to parse, and pytest understands it just fine.

1
2
3
4
5
import unittest

class YourModuleTests(unittest.TestCase):
    def test_add(self):
        assert 2 + 2 == 4

Running tests with pytest

Use pytest to run the tests. It integrates well with unittest, and there are a ton of useful plugins.

There are two plugins that every project should use:

When all is said and done, the command to run your tests should look something like this:

py.test tests/ --cov your_module --cov-append --cov-report term-missing --pep8

Automating tests with tox

tox is a tool that aims to automate and standardize tests. It's especially useful when you need to test code in different environments (i.e., multiple versions of Python).

tox is a much simpler command to remember than a long py.test command, so it's good to use even if you're only targeting a single environment.

A simple tox.ini looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
[tox]
envlist = py27,py35

[testenv]
commands=py.test tests/ --cov your_module --cov-append --cov-report term-missing --pep8
deps =
    pytest
    pytest-cov
    pytest-pep8
    -rrequirements.txt

Now when you run tox, you should see something like this:

$ tox

... a whole lot of output, including test output and coverage summary ...

_______________ summary _______________
  py27: commands succeeded
  py35: commands succeeded
  congratulations :)

Continuous Integration with CircleCI

Getting tests to run with tox on CircleCI requires the use of tox-pyenv to make different Python versions available.

In this example circle.yml, we'll run our tests and upload the coverage results to Codecov:

1
2
3
4
5
6
7
8
9
dependencies:
  override:
    - pip install tox tox-pyenv codecov
    - pyenv local 2.7.10 3.5.0

test:
  override:
    - tox
    - codecov