Run Python unittests in parallel

Overview

Thing you want to do: I want to execute test code in parallel using the Python standard unittest module.

How to do: Use nose as the test runner and specify the number of parallel executions with the --processes option

Confirmation environment

$ python --version
Python 3.7.2
$ nosetests --version
nosetests version 1.3.7

Sample code for confirmation

Since we want to see the effect of parallel execution here, let time.sleep () wait for 1 second in the test method.

$ cat test_s1.py
import unittest
import time
class Test(unittest.TestCase):
    def test_method(self):
        time.sleep(1)
        self.assertTrue(True)
$ cat test_f1.py
import unittest
import time
class Test(unittest.TestCase):
    def test_method(self):
        time.sleep(1)
        self.assertFalse("Hyaha")
$ ls
test_f1.py  test_s1.py
$ cp test_s{1,2}.py
$ cp test_s{1,3}.py
$ ls
test_f1.py  test_s1.py  test_s2.py test_s3.py

Use a standard test runner

It takes 4 seconds because it runs sequentially

$ python -m unittest
F...
======================================================================
FAIL: test_method (test_f1.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yoichi/prog/python-study/nose/test_f1.py", line 6, in test_method
    self.assertFalse("Hyaha")
AssertionError: 'Hyaha' is not false

----------------------------------------------------------------------
Ran 4 tests in 4.024s

FAILED (failures=1)

no setests option

$ nosetests
F...
======================================================================
FAIL: test_method (test_f1.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yoichi/prog/python-study/nose/test_f1.py", line 6, in test_method
    self.assertFalse("Hyaha")
AssertionError: 'Hyaha' is not false

----------------------------------------------------------------------
Ran 4 tests in 4.066s

FAILED (failures=1)

From https://nose.readthedocs.io/en/latest/plugins/multiprocess.html, the default is not parallel execution, so it is as expected.

--processes=NUM
  Spread test run among this many processes. Set a number equal to the number
  of processors or cores in your machine for best results. Pass a negative
  number to have the number of processes automatically set to the number of
  cores. Passing 0 means to disable parallel testing. Default is 0 unless
  NOSE_PROCESSES is set. 

Parallel execution with nosetests

If it is 2 parallel, it will end in 2 seconds

$ nosetests --processes=2
F...
======================================================================
FAIL: test_method (test_f1.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yoichi/prog/python-study/nose/test_f1.py", line 6, in test_method
    self.assertFalse("Hyaha")
AssertionError: 'Hyaha' is not false

----------------------------------------------------------------------
Ran 4 tests in 2.090s

FAILED (failures=1)

4 parallel finishes in 1 second

$ nosetests --processes=4
F...
======================================================================
FAIL: test_method (test_f1.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yoichi/prog/python-study/nose/test_f1.py", line 6, in test_method
    self.assertFalse("Hyaha")
AssertionError: 'Hyaha' is not false

----------------------------------------------------------------------
Ran 4 tests in 1.128s

FAILED (failures=1)

As the number of parallels increases, the number of resources used at the same time increases, so set an appropriate value according to the execution environment.

Unit of parallel execution

Multiple TestCases

Even if there are multiple TestCase classes in one file, they will be executed in parallel.

$ cat test_s1.py
import unittest
import time
class Test1(unittest.TestCase):
    def test_method1(self):
        time.sleep(1)
        self.assertTrue(True)
class Test2(unittest.TestCase):
    def test_method1(self):
        time.sleep(1)
        self.assertTrue(True)
$ ls
test_s1.py
$ nosetests --processes=2
..
----------------------------------------------------------------------
Ran 2 tests in 1.073s

OK

Multiple test methods

Does not execute in parallel on a method-by-method basis

$ cat test_s1.py
import unittest
import time
class Test1(unittest.TestCase):
    def test_method1(self):
        time.sleep(1)
        self.assertTrue(True)
    def test_method2(self):
        time.sleep(1)
        self.assertTrue(True)
$ ls
test_s1.py
$ nosetests --processes=2
..
----------------------------------------------------------------------
Ran 2 tests in 2.073s

OK