UpNext2 Development Record # 1 Build Python CI environment in VS Code

This is a continuation of [UpNext2 Development Record # 0] 1. This time, when creating preprocessing in Python, we will set up an environment that satisfies the goals C1 to C3 presented last time. The local environment is MacOS Catalina.

Goals C1 to C3 are reposted.

The following is a step-by-step overview of the setup.

1. Preparation of GitHub and cooperation with local repository handled by VS Code

After trying various things, the following procedure was the best for successfully linking local and remote.

• Create a new repository on GitHub side
• Create a develop branch on GitHub side
• Clone the repository created on the local side (a master branch is created on the local side and linked with the remote side)
• git checkout -b develop origin / develop to create a local develop branch and associate it with the remote
• Open local repository folder from VS Code

This makes it possible to perform staging to commit to the local repository and push to the remote repository by operating VS Code on the UI. (It may be necessary to set the user name etc. with the git command)

In addition, it is fun to include GitGraph etc. as an add-on for VS Code.

2. Set files not to be saved in the repository with .gitignore

Do not push work directories and files of various tools, work for development, and information that you do not want to publish (API key obtained by yourself, etc.) to the repository. The mechanism for that is .gitignore.

2.1. Creating a .gitignore file and defining its contents

Create a .gitignore file in the root directory of your project. The contents are as follows for now.

.gitignore

.*
*cache*
*local*
*secret*

In particular,. * And \ * cache * will be required. Without it, a large number of system work files and common libraries for python virtual environments would end up in the repository.

2.2. Don't push .gitignore itself

I like to do this, but it is possible to avoid pushing .gitignore itself.

Add excludesfile = .gitignore in the [core] group of .git / config in your project home.

3. Build a Python virtual environment with venv

In VScode, you can select and use the Python environment, but when preparing various things, let's make sure that Python 3 is used by default. Adjust the path so that 3 series appears as python -V. Make sure that pip is also 3 series. (Note that recent Python doesn't recommend typing pip directly, it seems to use it as python -m pip)

Since Python is complicated depending on the version of the library, it is recommended to prepare a virtual environment for each application and switch the library set. Previously, pyenv and virtualenv were used to switch to include the python version itself, but they are not recommended in Python 3.6 and later. It is now recommended to use venv.

In your project home, set up your venv environment as python -m venv .pyvenv. .pyvenv can be any other name, but it's best to start with. So that it's subject to .gitignore. Various common libraries are installed in the venv environment, so if you do not hit .gitignore, it will be a big deal. If you create a new venv environment, the library including pip will be in the initial state, so please re-install it according to various instructions.

After setting up the venv environment, you will be able to select the venv environment on VS Code, so please select it (at first, it does not appear even if you click the status bar, select it from Python: Select Interpreter in the command palette menu. Seems to need to). If you select it here, VS Code will automatically activate venv. Of course, you'll need to install VS Code's Python extension in the meantime.

4. Build a pytest environment

Currently, it seems that the library built-in pytest is often used instead of the Python built-in unittest. Select pytest from Python: Configure Tests in the command palette menu to build the environment. At this time, select .Root directory as the test directory.

To keep your sources and tests clean, divide your directories as follows: In addition, since we are planning to code other than Python in this plan, it is actually divided into subprojects.

(Project root)/
　├ src - temp.py
　└ tests - test_temp.py

Here, if you import temp.py from test_temp.py with a normal relative path specification, you will get addicted to an error.

By applying the updated solution of "[I wrestled with ModuleNotFoundError that occurs when I put the test code and the code to be tested in separate directories in Python] 2", I was able to solve it elegantly. It was.

The solution is:

• Create an empty \ _ \ _ init__.py file in the tests folder
• Import temp.py in test_temp.py as a relative path specification from the project home

By combining the above with the test directory as .Root directory in the pytest settings, you will be able to execute the test normally (Note that if you execute test_temp.py directly without using the VSCode UI, the import will fail. ). In addition, the correspondence based on VS Code is also posted in the reference link, this time it was done by trial and error.

5. Setting coding standards

In the command palette menu, under Python: Select Linter, select flake8. There may be various other settings, but I don't remember for a moment. Sweat

6. Set up automated test when committing local repository as CI

The coding standard check can be automatically checked at save time with VS Code, but the test must be executed manually. Originally, I want the test to be executed automatically when committing the local repository and when pushing to the remote repository. The latter is useful when the committer is different from the pull requester, and it seems that Travis CI used to be popular and GitHub Actions is used these days.

For personal development, it seems that only automatic testing at the time of committing the local repository is enough, so let's set it. It seems to be a good idea to use the hook function of the local git command to execute a test with the hook script of pre-commit. Create the project home /.git/hooks/pre-commit with execute permission as follows. Note that this script is outside the jurisdiction of VS Code, so don't forget to write activate.

pre-commit

#!/bin/sh
source .pyvenv/bin/activate
python -m pytest

Now when you commit to a local repository in VS Code, the test will run automatically and the commit will be canceled if the test fails. It's a success.