Python development helped by Jenkins-Unit test

Summary of this article

I will explain how to get Jenkins to do unit tests in development using Python. Nose or py.test are good candidates for Python's unit testing framework. This time, we will use py.test.

To Jenkins

• Automatic test execution
• Test report output
• Coverage output and visualization I have you do.

The environment is as follows.

py.test Install with pip install pytest

Test code example

Code to be tested --calculator.py

class Calculator():
    def add(self, a, b):
        return a + b

    def sub(self, a, b):
        return a - b

    def mul(self, a, b):
        return a * b

    def div(self, a, b):
        return a / b

Test code --test_calculator.py

import pytest
from calculator import Calculator

def pytest_funcarg__calc():
    return Calculator()

@pytest.mark.parametrize("a, b, r", [(9, 8, 17), (7, 6, 13), (5, 4, 9), (3, 2, 5), (1, 0, 1)])
def test_add(calc, a, b, r):
    assert calc.add(a, b) == r

@pytest.mark.parametrize("a, b, r", [(9, 8, 1), (7, 6, 1), (5, 4, 1), (3, 2, 1), (1, 0, 1)])
def test_sub(calc, a, b, r):
    assert calc.sub(a, b) == r

@pytest.mark.parametrize("a, b, r", [(9, 8, 72), (7, 6, 42), (5, 4, 20), (3, 2, 6), (1, 0, 0)])
def test_mul(calc, a, b, r):
    assert calc.mul(a, b) == r

@pytest.mark.parametrize("a, b, r", [(9, 3, 3), (8, 4, 2), (6, 2, 3), (4, 2, 2), (0, 1, 0)])
def test_div(calc, a, b, r):
    assert calc.div(a, b) == r

def test_div_error(calc):
    with pytest.raises(ZeroDivisionError):
        calc.div(1, 0)

Test run

Can be executed with py.test Recursively search under the executed directory and It runs py files that start with test.

test results Success story

============================= test session starts ==============================
platform darwin -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/rotasuke/git/python_test/others, inifile: 
collected 21 items

test_calculator.py .....................

========================== 21 passed in 0.03 seconds ===========================

Failure example

============================= test session starts ==============================
platform darwin -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/rotasuke/git/python_test/others, inifile: 
collected 21 items

test_calculator.py ...................F.

=================================== FAILURES ===================================
_______________________________ test_div[0-1-1] ________________________________

calc = <calculator.Calculator object at 0x10c9567f0>, a = 0, b = 1, r = 1

    @pytest.mark.parametrize("a, b, r", [(9, 3, 3), (8, 4, 2), (6, 2, 3), (4, 2, 2), (0, 1, 1)])
    def test_div(calc, a, b, r):
>       assert calc.div(a, b) == r
E       assert 0.0 == 1
E        +  where 0.0 = <bound method Calculator.div of <calculator.Calculator object at 0x10c9567f0>>(0, 1)
E        +    where <bound method Calculator.div of <calculator.Calculator object at 0x10c9567f0>> = <calculator.Calculator object at 0x10c9567f0>.div

test_calculator.py:21: AssertionError
===================== 1 failed, 20 passed in 0.04 seconds ======================

Jenkins job creation

Detailed settings such as Git checkout are omitted.

In [Build]-[Run Shell] py.test <directory to run> To specify.

Test report output

In the shell script to be executed that you entered earlier --junitxml = <destination xml file> By specifying, you can output the test report in JUnit format.

Added [Aggregate JUnit test results] to [Post-build processing] Specify the xml file that was output earlier.

Coverage output

Install the module for coverage output with pip install pytest-cov

In the shell script to be executed that you entered earlier --cov-report=xml --cov By specifying, coverage.xml can be output under the directory where the command is executed.

It can be visualized by having Jenkins' Cobertura Plugin read coverage.xml.