Overview

Manage the Python execution environment with Docker and Pipenv and use it as [VSCode] ](Https://code.visualstudio.com/) Remote Development introduces the mechanism to switch. News that Facebook has partnered with Microsoft for this feature is a hot feature these days. The benefits of introducing this are:

--Docker: Dependent libraries (LAPACK, etc.) and environment variable settings that cannot be managed by Pipenv can also be managed. --Pipenv: Packages required only for development (autopep8, etc.) can be managed separately from the execution environment, and dependency management is easier than pip. --VSCode: Can be seamlessly developed on the docker container built with the above two, and various extensions are powerful

I think that the merit of building a development environment by combining these is great. The image is composed as follows.

We also introduce recommended extensions and settings for developing Python with VS Code!

Version information

VSCode: 1.40.1
Docker: 18.09.7
docker-compose: 1.24.1

Preparation

--VSCode install: Quickly from Download page --Docker install: Community version is installed and [Non-root running settings](https://docs. docker.com/install/linux/linux-postinstall/)

I think that there are various articles that explain in detail other than the official one, so please check each one if the above is insufficient.

Environmental construction steps

Create a Dockerfile
Create a Pipfile
Describe the build method for the execution environment in docker-compose.yml
Install Remote Development extensions
Describe Remote Development specific settings in docker-compose.extend.yml
Describe the required settings in .devcontainer.json
Type a command with VS Code and build !!

I think that those who operate products using Docker or Pipenv have already made 1 to 3, so please skip to 4. It's a long way to go, but let's do our best to create a comfortable environment!

How to explain

Describe the configuration file with the following structure. I will explain the role of each file and how to create it in order. Replace {repo-name} with the repository name and {tool-name} with the root directory of your development tools.

{repo-name}
├── .devcontainer
│   ├── devcontainer.json
│   └── docker-compose.extend.yml
├── Dockerfile
├── Pipfile
├── docker-compose.yml
└── {tool-name}

Creating a Dockerfile

I'm ashamed to say that I'm not so familiar with Docker, so I think there are many things to do, but for example, create an environment in the following form.

`Dockerfile`


# Python 3.8 is slim-buster seems to be recommended
#reference: https://pythonspeed.com/articles/base-image-python-docker-images/
FROM python:3.8-slim-buster

#Set environment variables
ENV HOME /root
ENV TZ Asia/Tokyo
WORKDIR $HOME

#Install various libraries used in Python
# software-properties-common and wget are required for later clang 8 install
RUN set -ex \
  && apt-get update \
  && apt-get install -y g++ git openssh-client wget libblas-dev liblapack-dev gnupg software-properties-common make

#clang 8 install
#Since numba depends on llvmlite, install clang 8 series together
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - \
  && add-apt-repository "deb http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" \
  && apt-get update \
  && apt-get -y install clang-8 lldb-8 lld-8 gfortran

# llvm-Register the path to config
ENV LLVM_CONFIG=/usr/lib/llvm-8/bin/llvm-config

#host key information known_Described in hosts
#reference: https://gist.github.com/gregdeane/56a7499fddac7251f01dcc9bb64e8486
RUN mkdir -p -m 0600 ~/.ssh \
  && ssh-keyscan github.com >> ~/.ssh/known_hosts

#Pipenv install
RUN pip3 --no-cache-dir install pipenv

#Create a directory with the repository name you want to develop.
RUN mkdir {repo-name}

Since it is assumed that we will develop a private repository on GitHub this time, we will not clone the repository or install the dependencies in the Dockerfile. Docker-compose side does not yet support --ssh default, so install it by another method. If you develop in public repository, you should also do git clone and pipenv install --dev in Dockerfile.

Creating a Pipfile

While adding the necessary libraries to the execution environment, list the tools required for the development environment under [dev-packages]. For example

`Pipfile`


[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
Cython = "*"
joblib = "*"
numpy = "*"
scipy = "*"
scikit-learn = "*"
lightgbm = "*"

[dev-packages]
nose = "*"
coverage ="*"
flake8 = "*"
isort = "*"
Sphinx = "*"
sphinx-autodoc-typehints = "*"
autopep8 = "*"
setuptools = "*"
mypy = "*"

[requires]
python_version = "3.8"

It is like this. First of all, let's actually do docker run based on the above Dockerfile and check if this Pipfile works properly.

Introduction of Remote Development

Install based on Turorial of Remote Development. It can be easily installed from the VS Code GUI below.

It's easy.

Described how to build for operating environment in docker-compose.yml

Create a docker-compose.yml that builds the above Dockerfile. For Dockerfiles that require you to specify build-arg, set it here. The simplest example is

`docker-compose.yml`


version: "3.7"
services:
  dev:
    build:
      context: .
      dockerfile: Dockerfile

Is it a shape like?

Remote Development specific settings are described in docker-compose.extend.yml

It's the production from here. Create a docker-compose.extend.yml that consists only of the settings required for Remote Development. By passing two yaml files in .devcontainer.json created in the next section, the build will be done with the settings to which each setting is applied in order. For example

`yaml:docker-compose.extend.yml`


version: "3.7"
services:
  dev:
    volumes:
      - ~/.ssh/id_rsa:/root/.ssh/id_rsa:ro
    command: sleep infinity

Make it like this.

In the item volumes, the local public key is mounted on the Docker image used in Remote Development. Therefore, you need to register the public key of your local machine on GitHub. Please refer to here for details. Again, this is an unnecessary item because authentication is not required when developing with a public repository.

Also, command is mandatory this time. Without this, Remote Development would just build and die ...

Describe the required settings in .devcontainer.json

It's another breath when you come here. It is a file that specifies how to build the Dockerfile created above. Below is an example and the role of each parameter in the comments.

`json:.devcontainer.json`


{
    #Become the name of Workspace.
	"name": "{repo-name} DevEnv",

    #Root directory used to specify the location of the configuration file.
	"context": ".",

    # docker-List of paths to yml used in compose up.Strict order.
	"dockerComposeFile": [
		"../docker-compose.yml",
		"docker-compose.extend.yml"
	],

    #Docker with build settings for development environment-compse.service name on yml.
	"service": "dev",
	"shutdownAction": "stopCompose",

    #Root directory to be a VS Code Project.
	"workspaceFolder": "/root/{repo-name}",

    #Command to execute when Docker build is completed.Clone and install the repository here.
	"postCreateCommand": "git clone -b {branch name} {GitHub url} . && pipenv install --dev",

    #Settings to apply to the new VS Code launched in Remote Development.
	"settings": {
		"autoDocstring.docstringFormat": "sphinx",
		"autoDocstring.guessTypes": false,
		"editor.formatOnSave": true,
		"editor.suggestSelection": "first",
		"git.autofetch": true,
		"git.confirmSync": false,
		"kite.showWelcomeNotificationOnStartup": false,
		"python.formatting.autopep8Args": [
			"--max-line-length=200"
		],
		"python.linting.pylintEnabled": false,
		"python.linting.flake8Args": [
			"--max-line-length=200"
		],
		"python.linting.flake8Enabled": true,
		"python.linting.mypyEnabled": true,
		"python.jediEnabled": false,
		"terminal.integrated.inheritEnv": false,
		"vsintellicode.modify.editor.suggestSelection": "automaticallyOverrodeDefaultValue"
	},

    #If you register the ID of the extension you want to use in the Remote environment, it will be installed automatically when you build the environment..
	"extensions": [
		"ms-python.python",
		"njpwerner.autodocstring",
		"ms-azuretools.vscode-docker",
		"visualstudioexptteam.vscodeintellicode"
	]
}

To supplement the extension,

njpwerner.autodocstring A tool that automatically generates docstrings for functions and classes in Python. I want to automatically generate html for documents with Sphinx, so " autoDocstring .docstringFormat ":" sphinx " also sets the style of the docstring.

visualstudioexptteam.vscodeintellicode A super useful tool for Python that does very powerful completion etc. " python.jediEnabled ": false is a required setting.

python.linting I think it's a sin not to mention Type Hinting in Python3 series, so "python.linting.mypyEnabled ": true is mandatory, and it's hard to read that max-line-length is the default on current monitors. Since there is no help for it, " --max-line-length = 200 " is specified for each of ʻautopep8 and flake8`. The default linter in Python for VS Code is pylint, but when I used it, I had a lot of addiction around import, so [flake8](http:: //flake8.pycqa.org/en/latest/) is used.

It's like that.

Type a command with VS Code and build !!

(For Ubuntu) Open the command window with Ctrl + Shift + p on VS Code and hit ʻOpen Folder in Container ...` as shown below.

If you specify the directory where the local Dockerfile is located, build will start automatically and VS Code that runs on the environment built on Docker Container will be newly launched. At the first time, it takes time to build docker and pipenv install, so please wait patiently.

Finally, Python's Interpreter may not recognize the pipenv environment, but running reload window from the command window will fix it.

This completes the development environment! Thank you for your hard work! !!

[PYTHON] Build a comfortable development environment with VSCode x Remote Development x Pipenv