Static analysis of Python programs

Introduction

We have summarized the simple usage of the following useful tools available in Python.

What is pipenv

It makes it easy to manage packages and build virtual environments when developing with Python. It feels like npm in Node.js.

Preparation

  1. Installation of pipenv

    pip install pipenv
    
  2. Create pipfile

    pipenv --python 3.8 # 3.Part 8 can be changed to the version of Python you want to use
    

A Pipfile is created.

  1. Installation of development package Install a static analysis tool that is only used during development.

    pipenv install --dev black --pre
    pipenv install --dev flake8 --pre
    pipenv install --dev isort --pre
    pipenv install --dev mypy --pre
    

Each package is added to [dev-packages] of Pipfile, and Pipfile.lock is created / updated.

  1. Installation of production packages Install the package to be used for production.

    pipenv install requests --pre
    

Each package is added to [packages] of Pipfile, and Pipfile.lock is created / updated.

Environment

  1. Building a virtual environment Build based on the information in Pipfile.lock.

  2. For development environment

    ```bash
    pipenv sync --dev
    ```
    
  3. For production environment

    ```bash
    pipenv sync
    ```
    
  4. Update Pipfile.lock (OK when needed)

    pipenv install
    
  5. Enter the virtual environment

    pipenv shell
    
  6. Check the installation package

    pip list
    
Development environment results (click here)
```bash
Package           Version
----------------- ---------
appdirs           1.4.4
black             20.8b1
certifi           2020.6.20
chardet           3.0.4
click             7.1.2
flake8            3.8.3
idna              2.10
isort             5.5.3
mccabe            0.6.1
mypy              0.782
mypy-extensions   0.4.3
pathspec          0.8.0
pip               20.1.1
pycodestyle       2.6.0
pyflakes          2.2.0
regex             2020.9.27
requests          2.24.0
setuptools        49.6.0
toml              0.10.1
typed-ast         1.4.1
typing-extensions 3.7.4.3
urllib3           1.25.10
wheel             0.35.1
```

</div></details>
Production results (click here)
```bash
Package    Version
---------- ---------
certifi    2020.6.20
chardet    3.0.4
idna       2.10
pip        20.1.1
requests   2.24.0
setuptools 49.6.0
urllib3    1.25.10
wheel      0.35.1
```

</div></details>
  • You can see that the development package and its dependent packages are not installed in the production environment.
  1. Get out of the virtual environment

    exit
    
  2. Delete the virtual environment

    pipenv --rm
    

What is black

black is a Python code formatter that automatically modifies how to write programs. It also complies with the Python standard coding convention PEP8.

Setting

  1. Black setting You can define the settings in pyproject.toml. In the following, the length of one line is limited to 120 lines, and the folder to be not formatted is specified. If you want to make more settings, you can find out by checking.

    [tool.black]
    line-length=120
    exclude = '''
    /(
        .eggs
    | .git
    | .hg
    | .pytest_cache
    | .mypy_cache
    | .tox
    | .venv
    | build
    | dist
    )/
    '''
    
  2. Pipenv script settings Add scripts to Pipfile.

    [scripts]
    black = "black ."  # --If you add check, it will only be checked without formatting.
    

Run

pipenv run black

The python code will be formatted automatically

What is flake8

flake8 is a wrapper for the code check tool below.

--PyFlakes (pyflakes: code error checking) --pycodestyle (check if it conforms to pycodestyle: PEP8) --Ned Batchelder ’s McCabe script (mccabe: Cyclomatic complexity check)

  • Flake8 can only be checked and cannot be formatted.

Setting

  1. flake8 settings You can define the settings in setup.cfg. In the following, the length of one line is limited to 120 lines, and the folders that are not checked are specified. Also, E203, W503, W504 are ignored in consideration of black. If you want to make more settings, you can find out by checking.

    [flake8]
    exclude = .git, .tox, .venv, .eggs, build, dist, docs, tests
    max-line-length = 120
    ignore = E203,W503,W504
    
  2. Pipenv script settings Add scripts to Pipfile.

    [scripts]
    flake8 = "flake8 ."
    

Run

pipenv run flake8

The python code is checked.

What is isort

isort sorts the imports alphabetically and automatically divides them into sections.

Setting

  1. setting isort You can define the settings in setup.cfg. In the following, the length of one line is limited to 120 lines, and the folders that are not checked are specified. If you want to make more settings, you can find out by checking.

    [isort]
    line_length = 120
    skip = .git, .tox, .venv, .eggs, build, dist, docs
    include_trailing_comma = true
    multi_line_output = 3
    '''
    
  2. Pipenv script settings Add scripts to Pipfile.

    [scripts]
    isort = "isort . --atomic" # --If you remove atomic, it will only be checked without formatting.
    

Run

pipenv run isort

The python code will be formatted automatically

What is mypy

mypy will check if you can write Type Hints that conforms to PEP484.

Setting

  1. Mypy settings You can define the settings in setup.cfg. The following ignores the lack of a module type definition file (stub) and forces type annotations. If you want to make more settings, you can find out by checking.

    [mypy]
    ignore_missing_imports = True
    disallow_untyped_defs = True
    
  2. Pipenv script settings Add scripts to Pipfile.

    [scripts]
    mypy = "mypy ."
    

Run

pipenv run mypy

The python code is checked.

Summary

With these tools, you'll be more productive in developing with Python. If you manage your code on GitHub, set the following in GitHub Actions and it will automatically perform static analysis when you push. If you get an error, fix it and push again.

.github / workflows / ci.yml (click here)

yml:.github/workflows/ci.yml


name: Source Code Check
on: [push]
jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@master
      - name: Setup Python
        uses: actions/setup-python@v1
        with:
          python-version: '3.8'
      - name: Install Python Dependencies
        run: |
            python -m pip install --upgrade pip
            pip install pipenv
            pipenv sync --dev
      - name: Check with black
        run: pipenv run black
      - name: Check with flake8
        run: pipenv run flake8
      - name: Check with isort
        run: pipenv run isort
      - name: Check with mypy
        run: pipenv run mypy

Recommended Posts