[No venv required] The strongest Python development environment created with Remote Containers [VS Code / Docker]

Introduction

Python is attracting attention as it shines in the top 1 popular language [^ 1] and becomes an exam subject for the IPA Fundamental Information Technology Engineer Examination [^ 2].

But wait a minute! Are you developing properly benefiting from the Python ecosystem?

By creating a good development environment, you can not only improve the efficiency of development ** by input interpolation and formatter, but also ** prevent unexpected problems by static analysis **, ① Easy ② Explosion velocity ③ You can write the program safely.

Of course, there are many How to articles on how to create such a comfortable environment, but the problem is that even if you can create an oleore Python development environment, it is difficult to share it between different PCs.

Because

• You have to install the same version of the module to be installed with Python or pip.
• You have to install and configure fommter / linter / IDE extensions

This is because there is a problem that depends on the development environment. ** I want you to be assured that the same operation will be performed between different PCs, whether individual or multiple people **. In addition, there is a request that ** just open the project with a text editor and the development environment should be built without permission **.

So, in this article, I'll solve this problem by using VS Code's Remote Container. Using the methods described in this article, you can:

• You can create a portable Python development environment that is guaranteed to work the same in all environments regardless of Win / Mac / Linux [^ 3].
• Even in a new environment, all settings such as static analysis and automatic formatting are completed just by opening the project.

Let's do it

Installation

In this article, we will use the following tools. Please install while referring to other articles.

• VSCode
• Docker

If you can type the following command from the terminal, you are ready to go.

$ docker --version
Docker version 19.03.5, build 633a0ea

Remote Containers settings

First, create a development directory and open it with VS Code.

$ mkdir python-test

Create the following directory structure

.
└── python-test
    ├── .devcontainer
    │   ├── Dockerfile
    │   └── devcontainer.json
    ├── .vscode
    │   └── extensions.json
    └── src
        └── main.py

Please copy and paste the following contents into each file.

.devcontainer/Dockerfile

FROM python:3.7.3-slim

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
    apt-utils \
    gcc \
    build-essential \
    && pip install --no-cache-dir \
    autopep8 \
    flake8 \
    && apt-get autoremove -y \
    && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/*

Here, you can specify your favorite Python version by changing the number in the FROM python: 3.7.3-slim part.

json:.devcontainer/devcontainer.json

{
    "name": "Python Project",
    "dockerFile": "Dockerfile",
    "settings": {
        "terminal.integrated.shell.linux": "/bin/bash",
        "python.pythonPath": "/usr/local/bin/python"
    },
    "extensions": [
        "ms-python.python"
    ]
}

Here, you can replace the " name ":" Python Project " part with any string you like.

In settings, you can write the same thing you write in .vscode / settins.json. Originally, the bash path and python path are environment-dependent, so it is necessary to set them for each individual, but this time it is executed in the container, so the path information is known and can be fixed. Is the point.

You can also add VS Code extensions to ʻextensions`, and the extensions described here will be installed automatically. Only Remote Containers can force VS Code extensions to be installed, so write the extensions you want everyone to install here. This time I'm adding a Python extension for VS Code.

src/main.py

import sys
print(sys.version_info)

Let's assume that the Python script outputs the version.

json:.vscode/extensions.json

{
    "recommendations": [
        "ms-vscode-remote.remote-containers"
    ]
}

Specify Remote Containers for Recommendations for VS Code extensions. It's important to note that unlike extensions in devcontainer.json, writing them doesn't actually install them. So, when you've copied and pasted the file, reopen VS Code for the actual installation. Then, the following pop-up will appear in the lower right corner. Select Install All to install the Remote Container.

Once you've successfully installed it, you'll see a green button at the bottom left of VS Code.

When you open it, you will see a menu like this,

Select "Remote-Containers: Open Folder in Container ...". When the folder selection screen appears, select the project directory (python-test in this case).

Then, the build of the container described in .devcontainer / Dockerfile will run automatically. You can see the progress by opening details. If you get stuck in the build, take a look at the details.

When the build is completed normally, it will start the container and connect to it. Once connected to the container, open VS Code's internal terminal.

Oh? Somehow the shell user is root and the directory is also / workspaces. This indicates that you are putting it in a container terminal that is independent of the host OS, not in the terminal of the host OS (Win / Mac, etc.).

I don't know Docker! You might say that, but let's run Python for now.

/workspaces/python-test# python src/main.py 
sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)

You have the version of python specified in FROM python: 3.7.3-slim! This completes the basic settings.

From now on, even if you git clone on another PC, you can reproduce the same environment by entering the container with" Remote-Containers: Open Folder in Container ... ".

Settings in requirements.txt

This is inside a container, so you can do any pip install inside it. The modules installed here have no effect on the host OS, so development can proceed destructively.

/workspaces/python-test# pip install numpy
/workspaces/python-test# python
>>> import numpy as np
>>> np.__version__
'1.17.4'

Also, if you build the container again and recreate it, the installed modules will disappear. So, let's fix this pip module as we proceed with development! If so, let's create requirements.txt, which is customary in Python, and write a module to pip install in it.

The directory structure and how to write requirements.txt are as follows.

.
└── python-test
    ├── .devcontainer
    │   ├── Dockerfile
    │   └── devcontainer.json
    ├── .vscode
    │   └── extensions.json
    └── requirements.txt

requirements.txt

numpy==1.17.4

In requirements.txt, you can write a fixed version for pip install. To ensure reproducibility, it is desirable to fix the version.

And let's make devcontainer.json as follows.

json:.devcontainer/devcontainer.json

{
    "name": "Python Project",
    "dockerFile": "Dockerfile",
    "settings": {
        "terminal.integrated.shell.linux": "/bin/bash",
        "python.pythonPath": "/usr/local/bin/python"
    },
    "extensions": [
        "ms-python.python"
    ],
    "postCreateCommand": "pip install -r requirements.txt"
}

You've added postCreateCommand. This is the command to run after the container is created. From now on, the modules described in requirements.txt will always be installed.

After updating the file, press the green button at the bottom left and select "Remote-Containers: Rebuild Container" to reflect the update in the container.

Set formatter / linter

Next, set formatter / linter. The devcontainer.json when adding the python formatter / linter is as follows.

json:.devcontainer/devcontainer.json

{
    "name": "Python Project",
    "dockerFile": "Dockerfile",
    "settings": {
        "terminal.integrated.shell.linux": "/bin/bash",
        "python.pythonPath": "/usr/local/bin/python",
        "python.linting.pylintEnabled": false,
        "python.linting.flake8Enabled": true,
        "python.linting.flake8Args": [
            "--ignore=E402,E501"
        ],
        "python.formatting.provider": "autopep8",
        "python.formatting.autopep8Args": [
            "--ignore",
            "E402,E501",
            "--max-line-length",
            "150"
        ],
        "[python]": {
            "editor.formatOnSave": true
        }
    },
    "extensions": [
        "ms-python.python"
    ],
    "postCreateCommand": "pip install -r requirements.txt",
}

flake8 is specified as linter and autopep8 is specified as formatter, and E402 (Module level import not at top of file) and E501 (Line too long) are ignored, respectively. Format is applied when saving a file.

Even if you write such a messed up code with this,

before.py

a    =0
def method1(args)  :
    print(   args)
def method2() :
    if a    ==0:
            method1(   "hoge"   +    "fuga"   )

In this way, it will be formatted when you save it.

after.py

a = 0


def method1(args):
    print(args)


def method2():
    if a == 0:
        method1("hoge" + "fuga")

Normally, such formatter / linter settings are described in .vscode/settings.json. However, since .vscode/settings.json is a personal configuration file, many projects have it set to .gitignore. In addition, you have to run pip install flake8 autopep8 manually, so writing the config file doesn't mean that formatter / linter will be applied.

On the other hand, this time it can be written in the Remote Containers configuration file, so it is easy to share, and since it can surely reproduce the state where flake8 and autopep8 are included, "Unformatted source code is committed" You can also prevent that.

By the way, where was pip install flake8 autopep8 running? The answer is in .devcontainer/Dockerfile.

.devcontainer/Dockerfile

FROM python:3.7.3-slim

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
    apt-utils \
    gcc \
    build-essential \
    && pip install --no-cache-dir \
    autopep8 \
    flake8 \
    && apt-get autoremove -y \
    && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/*

If you want to write pytest inside the container, you can add it.

.devcontainer/Dockerfile

RUN pip install --no-cache-dir \
    autopep8 \
    flake8 \
    pytest

Also, autopep8 and flake8 are required at development time, not at runtime. You can write such modules in Dockerfile and write the files required at runtime in requirements.txt.

Static type analysis

Finally, let's introduce Type Hints that can be used with Python 3.6 or later [^ 4]. By introducing Type Hints, QoL will explode.

Let's make devcontainer.json as follows Here, pyrite is used as a static analysis tool for Type Hints.

json:.devcontainer/devcontainer.json

 "extensions": [
    "ms-python.python",
    "ms-pyright.pyright"
 ],

After making changes, press the green button at the bottom left and select "Remote-Containers: Rebuild Container".

Next, write the pyright configuration file. The directory structure is as follows.

.
└── python-test
    ├── pyrightconfig.json
    └── src
        └── main.py

pyrightconfig.json

{
    "include": [
        "src"
    ],
    "reportTypeshedErrors": false,
    "reportMissingImports": true,
    "reportMissingTypeStubs": false,
    "pythonVersion": "3.7",
    "pythonPlatform": "Linux",
    "executionEnvironments": [
        {
            "root": "src"
        }
    ]
}

Please refer to other articles for details on how to write pyrightconfig.json. (Maybe I write something else)

Now let's edit src / main.py.

src/main.py

def hello(name: str, age: int) -> str:
    result1: str = "My name is " + name + ".\n"
    result2: int = "I am " + str(age) + " years old."
    return result1 + result2


result: int = 10
result = hello(name="Otao", age=23)
print(result)

Type Hints in python allows you to annotate functions and variables in this way. If you open src / main.py with pyright installed, ... Since result2 is an int type, str type cannot be assigned! The str and int + operators are undefined! !! result is defined as int type, but I'm trying to assign str type to it! !! !! I'll also post the definition of the hello function! !! !! It will display the error related to the type.

We recommend using Type Hints for the following reasons:

• Defining a type messes up readability
• Reduces the chance of unexpected types entering variables and causing problems
• Input completion function on VS Code works

Let's introduce it.

Well, strangely (?) This script can be executed, isn't it? (Type Hints is just an annotation, so it is ignored at runtime.)

/workspaces/python-test# python3 src/main.py 
My name is Otao.
I am 23 years old.

At the end

Have a comfortable Python life! (Articles will be updated accordingly.)

• Author information
• Twitter: @0meo

[^ 1]: Programming languages you want to learn, languages you don't want to learn, https://tech.nikkeibp.co.jp/atcl/nxt/column/18/00501/111200004/ [^ 2]: Press release: Review the questions in the Fundamental Information Technology Engineer Examination, https://www.ipa.go.jp/about/press/20190124.html [^ 3]: Actually, the behavior may differ depending on the host OS, and it may be necessary to modify the Dockerfile. [^ 4]: Type Hints for functions was from 3.5, so it's not accurate, but variables can also be type-annotated since 3.6, so I'd like you to use 3.6 or later.