A story about a salaryman engineer who works for a company that does not have a test code culture, and works hard to spread the test culture while asking the manager's complexion. At the end of the article, I write a perspective on code review of tests.
"This is a release-first project. The idea of writing test code as a team in one word from the manager was rejected. I could understand why I shouldn't go on an adventure because I don't have the know-how to write test code in-house. However, it should be okay to write a test in my spare time, so I decided to do my best.
When I asked an engineer I knew about it, there were about two people who had written a test. I won't tell him, but he will be certified as a mentor and will teach you efficient methods and know-how in Python and Django. Looking back, I think I was blessed with the environment. There are multiple Python test frames, but the official documentation is solid Pytest. Pytest is highly functional and the documentation is complicated, so I think you can learn how to use it with Examples and use it as a dictionary.
Install pytest
pip install pytest
pytest test sample
# -*- coding: utf-8 -*-
from __future__ import absolute_import, unicode_literals
#Function to test
def add(a, b):
return a + b
#Test code function name is test_How to start with pytest
def test_add():
assert add(1, 1) == 2
assert add(1, 2) != 2
Execution result of pytest
>>> $ py.test ../tests/test_add.py
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 2.7.5 -- py-1.4.31 -- pytest-2.7.0
rootdir: /Users/***********, inifile: pytest.ini
plugins: cache, django, pep8, pythonpath
collected 2 items
../tests/test_add.py ..
============================================================================ 2 passed in 0.06 seconds =============================================================================
At first, I worked hard to write unit tests, but the burden on developers is heavy. There is a feeling of trying too hard. When I consulted with him, he advised me to do a black-box test on a Web API basis instead of a function-based test on a social network server. When I tried it, the development man-hours were small, and it was simple and powerful. I still use it. Code coverage did not improve, but Web API testing can cover the API with less effort.
This test method is especially useful in scripting languages, where you will often find the kind of bugs that are detected at compile time.
Pytest code to test JSON-style Web API
# -*- coding: utf-8 -*-
from __future__ import absolute_import, unicode_literals
import ujson
import requests
def test_api():
#Test the GitHub API
url = "https://api.github.com/repos/vmg/redcarpet/issues?state=closed"
headers = {'Accept-Encoding': 'identity, deflate, compress, gzip',
'Accept': '*/*', 'User-Agent': 'python-requests/1.2.0',
'Content-type': 'application/json; charset=utf-8',
}
response = requests.get(url, headers=headers)
#HTTP Status code is 200
assert response.status_code == 200
#Being able to parse BODY with json
data = ujson.loads(response.text)
#Url in array, state, created_The existence of the element of at
for param in data:
assert 'url' in param
assert 'state' in param
assert 'created_at' in param
pytest execution result
>>> $ py.test ../tests/test_webapi.py
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 2.7.5 -- py-1.4.31 -- pytest-2.7.0
rootdir: /Users/*****, inifile: pytest.ini
plugins: cache, django, pep8, pythonpath
collected 2 items
../tests/test_webapi.py ..
============================================================================ 2 passed in 0.87 seconds =============================================================================
When all the API tests of the server were completed, we made a rule to consult within the team and write the Web API test. Since pytest has a simple mechanism, anyone with sample code could easily learn it. However, it takes a long time to do muddy work such as pointing out in the review until it is fixed, executing tests regularly and fixing any errors, and telling new participants. (If you are familiar with it, run it regularly? I think you felt that it was an automatic test with commit hook. At that time, I was thinking of introducing it, but I forgot to introduce it without getting heavy. )
-Enabled to execute all tests with one command (written by shell) ・ In order to develop comfortably, all tests were executed within 60 seconds. ・ It was decided that the test would pass the work completion condition in the local environment construction procedure. ・ Confirmed that the test passed during code review ・ When developing a new function, it is a prerequisite that a test is attached at the time of code review. ・ I ran a test on a regular basis and had them fix it or fix it.
The app was successfully released and a test culture was taking root within the team. Development is done on a pull request basis, with test code attached without pointing out. Due to the test failure, we found a dependency between the newly developed module and the module that was thought to be out of scope, and we were able to prevent production bugs.
As an operation rule, the checklist at the time of deploy always clearly states that test execution is performed, so the habit of executing tests from the development stage has taken root among the members.
I held an in-house study session and told him that it would be good to write a test. The reason why our project was successful (one of them) was that I wrote test code. Especially in the company, it was effective because I could share the actual test code.
It's a test of time, and if you continue to operate it for a long time, the test code will carry technical debt. A problem child test that needs to be corrected every time you make a change, a test that takes more than 300 seconds to execute once, and a test code that is left commented out because an error occurred due to a problem with the test code itself when deploying in a hurry. Test code that is too complicated to fix and you don't know what you are testing.
It has been one year since the operation started. Tests that initially completed in less than 60 seconds now take 12 minutes to complete. The rule to run the test on deploy is also omitted due to latency. Many of the engineers were veterans and the debugging team was excellent, so the bugs did not increase. Many people wrote tests for the new module pull request, but the test culture within the project has declined.
Due to the influence of the in-house study session and the fact that some engineers had already written the test code individually, although I intended to start it alone, the number of new projects that gradually adopt the test writing as a rule is increasing. went. I'm doing a test! I think it is important to say that.
end
If you neglect to implement the business logic, it will not work according to the specifications, so it is essential to implement it, but it is quite good even if you do not implement or execute the test code. I think the reason why the test culture has declined is that the development team is exhausted from the maintenance of the test, so I will write down where it was exhausted.
I think we should have linked commit hook and jenkins, or adopted Travis CI or Circle CI and automatically tested at the timing of commit. I wrote about how to use CircleCI in Django in this article
The team was exhausted by the maintenance of the test, which is complicatedly dependent on master data and fails once every two times due to the problem of the test itself. I think that poor quality tests should have been thrown away and rewritten for metabolism.
I can't wait for a test that takes more than 60 seconds. Since the tempo of development will be slow, I think that we should have considered more consideration so that we can develop comfortably by considering parallel execution of tests and dividing it into two layers such as quick test and full test.
Although an error will occur if it is operated for a long period of time, the number of variables are lined up and the person who wrote it does not know what test it is. I think it was.
This is an effective measure for new participants. Even after construction, if you check whether the test passed for each pull request, it will be fixed quickly.
If you proceed with development based on pull request, control will be effective, so you can prevent the situation where the number of untested code increases, and the educational effect is great.
When a test failure occurs, no one will see the test result, so we will give priority to fixing it. If someone fixes it on a full-time basis, it will not improve forever, so it may be good for education if you carry it around on duty.
If the test execution time is slow, you will not be able to see the results, so you should take steps to execute the tests in parallel or speed up.
This is the point of view to check in the code review when receiving a pull request.
Automate or actually check out to make sure the test passes.
Unless you have a specific reason, one test should be completed within 5 seconds. Check for any tests that take a long time in terms of execution time.
Fixed values such as card_id = 1 are technical debt that will result in an error if the ID is changed in the future. Insist that you load the test value in fixtures instead of a fixed value, or pull the ID from another data acquisition API.
Check in terms of what perspective the test is described in. After a long period of operation, the members will be replaced, so even if an error occurs in the test, the reason will not be clear.
If you write a separate test, for example, register and then delete, a dependency will occur. If there is a dependency on the execution order of the tests, an error will occur when parallelizing the tests for speed, so if there is a dependency, point out that it should be combined into one.
Recommended Posts