How to build a Python virtual execution environment using Visual Studio Code and pipenv on a Windows machine (also Jupyter notebook)

at first

This article is a forcible drop of the laboratory materials originally created in PDF into Markdown. Therefore, there are problems such as "unnecessarily itemized", "header structure is not good", "too many figures for Web document", etc., and it is far from "concise and easy-to-understand Qiita article". .. Please understand that point.

In addition, the story of environment construction here will soon become old information, and there is a possibility that the method as written will not work in the future. I am not responsible for any problems that may occur with reference to this article, so please use this article as a reference only and take full responsibility for trying it out. In addition to this article, there are many Web pages that explain similar content, so you may want to refer to that as well.

Purpose of this article

--Explanation of building an environment where Python can be executed on a Windows machine using only VSCode (editor) and Python (interpreter) without using Anaconda. --Do not use the famous Python environment installer Anaconda --The reason for not using it will be described later. --Explanation of how to build a directory-complete virtual environment using pipenv --Since the packages to be installed are managed on a directory basis, development can be done without polluting the execution environment of the PC itself. --Explanation of how to build an environment to run Jupyter notebook --Jupyter notebook is a famous editing environment for beginners who can execute and check code on a cell-by-cell basis.

About python

Features of Python

--High-level general-purpose scripting language (executed line by line) --Dynamic typing --Remarkably popular in the fields of machine learning and deep learning --Abundant library, abundant information

Execution environment construction is often troublesome

――This document introduces the procedure for building an environment that executes Vanilla Python from Visual Studio Code, a high-performance editor. ―― "For the time being, Anaconda is easy but dangerous" --Anaconda is a batch installer for Python environment (from IDE to package) --Anaconda's original virtual environment package management software "conda" can cause confusion if it coexists with Python's genuine package management software "pip". --You only need to use conda, but there are packages that can only be installed with pip. --When you install a calculation package called numpy with conda, the linear algebra library (BLAS) becomes Intel MKL instead of OpenBLAS, which is slow

About Visual Studio Code and Python

VS Code Features

--General-purpose high-performance text editor --Development is Microsoft --Abundant extensions make it suitable for all text descriptions --C, C ++, Java, HTML, CSS, MATLAB, Python, etc ...

VSCode＋Python

--Python integrated development environment (IDE) features --Run / Debug --Isn't it the simplest Python development environment?

How to build VSCode + Python execution environment

VS Code installation

download

• https://code.visualstudio.com/download/ --Download according to each platform (64bit version)

Installation

--Double-click the downloaded installer --Agree to the license agreement and click "Next" --Leave the default path "Next" --Keep the default program group settings and click "Next" --Set the selection of additional tasks as you like and click "Next" --Adding [Open with Code] to the context menu is convenient because it can be started by right-clicking. --Final confirmation and "Install" --VSCode starts automatically after installation --Close once and restart Windows

Extension installation

--The icon at the bottom left of the screen is related to extensions

Japanese localization extension

--Enter "Japanese" in Search extensions and click "Install" --Click Restart Now at the bottom right of the screen after installation to restart --After restarting VS Code, it will be translated into Japanese

Recommended settings for VS Code

Turn off the Hot exit function

--Ability to exit VS Code even if the file is not saved (source of confusion) --Enter "hot exit" in "Search settings" on the Settings tab and set it to "off"

Python installation

download

• https://www.python.org/downloads/ --Download according to each platform --It is safer to set the Python version to 3.6.10 or 3.7.6 (as of March 2020) --New and Python libraries and packages may not be compatible --Any x86-64 (64bit version) is fine, but select the executable installer to download --For Linux and Mac, select the dedicated one

Installation

--Double-click the downloaded installer --Click "Customize installation" --"Next" with all items checked --Select "Install for all users", "Set Python path as environment variable" and click "Install" --If you do not install it for all users, the path will not pass with pipenv) --"Close" after installation is complete

VS Code + Python environment construction / execution

Actually create a Python script and execute it

--Create a new folder "test", right-click and "Open with Code" --Install extensions for Python --Return to file management and create a file by selecting "File"-> "New File" --A new file called "Untitled-1" is created --Save as "test.py" by selecting "File"-> "Save As" ――Let's write a program appropriately

test.py

import os
path = os.getcwd()
print( "The current directory is" + path + "is" )

--Save and press F5 (Run) to bring up the debug menu --Click "Python File" (or enter as it is) --The terminal (Windows PowerShell) is displayed, and the Python execution command ("python ./test.py" -like command) is executed on it and the result is displayed. --Congratulations (moved)

VS Code + Python debugging

Try debugging while running

--Create a new Python script "inc.py", enter the following and save ("increment" in the image is misspelled and becomes "incriment")

inc.py

def increment(n):
    return n+1

a = 3
b = increment(a)
print(a)
print(b)

--The result is displayed when you execute with F5 → Enter. --Click on the left side of the line number in the editor to set a breakpoint --F5 → If you execute with enter, it will be executed until just before the breakpoint

Other information on VS Code + Python

--Launching a new terminal for VS Code --"Terminal"-> "New Terminal" on the menu bar --Delete the past character string (command log) of the terminal --Enter "clear" --Interpretered Python interpreter --Type "py" on the VS Code terminal (Windows PowerShell) to launch the interactive Python interpreter ――Since it can be executed line by line, it is convenient for a little confirmation etc. --Interactive mode ends with "exit ()"

How to build a virtual execution environment with pipenv

What is a virtual execution environment?

--Python can use more functions by importing an additional library called "package". --Numpy for numerical calculation, pandas for data manipulation, etc. --If you declare "import numpy as np", you can use numpy functions in the form of np.xxx. --Same concept as reading a header file with #include in C language --In Python, install / uninstall each package using package management software called "pip". --Run "pip install numpy" in the terminal to install numpy --Since the version described later is not specified, the latest version of numpy will be entered. --If you execute "pip list" in the terminal, the list of currently installed packages will be displayed. --You can uninstall numpy with "pip uninstall numpy" --Since Python is open source software, additional functions will be developed one after another for each package, and the version will be upgraded. --Some packages refer to the features of other packages (that is, they don't work unless both packages are imported), so if you're unlucky with upgrading one package, other packages may stop working. --Example: It works in the execution environment where "version 1.14.1 numpy" and "version 0.22.0 pandas" are imported into "version 3.6 Python"! ――In other words, "It may not work in other execution environments!" --Execution environment can be output if the package is installed with pip --You can output the installed package with "pip freeze" as a text file called requirements.txt. --If you want to reproduce the execution environment on another PC, copy the above text file and execute "pip install –r requirements.txt" to install all at once. --If you install pip when you need the necessary packages, the number will increase (200 packages, etc. if you notice). --I don't know the minimum package required for the project --If you can easily switch the execution environment (Python itself and the version of each package), it is convenient to be able to quickly build the execution environment of others without destroying your current execution environment. --The concept of "virtual environment" has appeared --Manage and build the execution environment "virtually" ――For example, you can manage the installation of Python and each package in a directory instead of the PC. --Since the environment changes if you change the directory, you can build a different execution environment for each project on the same PC. --When moving to another PC, you can pass each directory (or package management information). --Package management software for virtual environment is in the Warring States period (too much) --"Conda (Anaconda environment only)", "virtualenv", "venv", "pipenv", ... --This article introduces probably the most standard "pipenv" virtual environment construction. --Only "pipenv" is installed on the PC with pip --Others are installed by pipenv for each directory (project)

Installation and initial setup of pipenv

Installation

--Launch VS Code --"Terminal"-> "New Terminal" on the menu bar --Enter "pip install pipenv" and execute --Various characters appear and pipenv is installed --Check with "pip list" --In addition to pipenv, some necessary packages are included.

Initial setting

--pipenv stores virtual environment files (installed package information) under the user home by default. --Change environment variables if you want to create a virtual environment for each directory (project) --"Control Panel"-> "System and Security"-> "System"-> "Advanced system settings"-> "Environment variables" on the left menu --Change environment variables if you want to create a virtual environment for each directory (project) --Click "New" in the "System Variables" column --Variable name is "PIPENV_VENV_IN_PROJECT" and value is "true". --Check the directory settings of VS Code's virtual environment just in case --VS Code menu bar "File"-> "Basic Settings"-> "Settings" --Enter "python.venvPath" in "Search settings" and if the item that appears is ".venv", it's OK.

Building and executing a virtual environment with pipenv

Try installing and running Python in a virtual environment

--Create a new folder "envTest", right-click and select "Open with Code" --Start a new terminal and run "pipenv --python 3.7" --However, 3.7 python must be installed on the system --Create a new file "test.py" in "envTest" and open it with VS Code --You will be prompted to choose which Python to use --Select an interpreter for the virtual environment (directly under the directory) --Of course, you can also use Python installed on your PC. --Ready to run in virtual environment --Hello world (F5 → Enter)

Try adding a package to your virtual environment

--Run "pipe nv install numpy" to install numpy --You can also specify the version like "pipenv install numpy == 1.16" --Execute a script that uses numpy --I get an error if numpy is not installed ...?

test.py

import numpy as np

A = np.array([ [1,2], [3,4] ])
B = np.array([ [0,1], [1,0] ])
C = A@B
print(f"Matrix C is the matrix product of A and B, and the result is\n{C}\n.")

Try removing the package from the virtual environment

--Uninstall from virtual environment with "pipe nv uninstall numpy" --Pipfile and Pipfile.lock are also rewritten --Execute a script that uses numpy --I get an error if numpy is not installed ...?

Reproduction of execution environment from other environment

--Environment reproduction from requirements.txt --Put requirements.txt in the project directory and execute "pipenv install –r ./requirements.txt" --Environment reproduction from Pipfile or Pipfile.lock --Pipfile in the project directory and "pipenv install" or "pipenv install --dev" (also install the development package) --Pipfile.lock in the project directory and "pipenv sync" or "pipenv sync --dev" (install development package as well)

List of packages installed in the virtual environment

--View Pipfile with cat or text editor --Pipfile.lock contains details such as the version actually installed for the package installed without specifying the version.

important point

--When you open a Python file in a directory with a virtual environment with VS Code from the right-click menu, you cannot select the virtual environment interpreter. --You can select the virtual environment interpreter by first opening the directory with VS Code instead of the file, and then opening the Python file with VS Code.

How to build an execution environment for Jupyter notebook

What is Jupyter notebook?

Jupyter notebook (Jupyter notebook)

--Write a Python script in a file in the form of a notebook --Easy to understand because it is possible to execute cell by cell and draw plots. --You can write a script and execute it immediately to check the result. --Useful for beginners learning Python and research experiment history --Convenient for code sharing ――It used to be called iPython notebook --The notebook file has its remnants in the extension (.ipynb)

Jupyter notebook system

--Genuine Jupyter notebook can edit and execute notebooks by setting up a local (or remote) server and connecting with a browser. --From October 2019, VS Code's Python extension (installed on p.27) natively supports Jupyter notebook --In this article, we will explain how to install Jupyter notebook in a virtual environment created with pipenv and create and execute a notebook file.

Install Jupyter notebook

Try installing Jupyter notebook in a virtual environment

--Create a new folder "jupyTest", right-click and "Open with Code" --Start a new terminal and run "pipenv --python 3.7" --Install Python 3.7 in virtual environment --Next, run "pipe nv install jupyter" in the terminal --Install Jupyter notebook in virtual environment --It will take some time because many necessary packages will be installed.

Try running Jupyter notebook in a virtual environment

--Press Ctrl + Shift + P to display VS Code's "Command Palette" --Enter "Python: Select" in the command palette and click "Python: Select Interpreter to start Jupyter server" that is displayed. --Select Python for the virtual environment directly under jupyTest --Note that you may also see Python in other directories remaining in the cache (check the path carefully and select it). --OK if the interpreter display at the bottom left of the screen becomes Python of jupyTest --If the interpreter display at the bottom left remains "Select Python Interpreter" even after selecting it, restarting VS Code may work. --Press Ctrl + Shift + P to display VS Code's "Command Palette" --Enter "Python: Create" in the command palette and click "Python: Create New Blank Jupyter Notebook" that appears. --A new notebook is created --Save as "test.ipynb" with Ctrl + S --The extension ipynb is the file extension of Jupyter notebook. --Write a script and execute the cell with Shift + Enter --The execution result is displayed just below the cell --Note that Jupyter variables span multiple cells, not cell by cell.

Finally

Python is a simple language, but avoiding Anaconda makes it difficult to build an environment. Also, if you search on the Web, only Anaconda articles will appear, so it is troublesome to do a negative search and collect information. As of March 2020, VSCode is being developed by Microsoft, and we can expect further development in the future. Therefore, I think that the execution environment mentioned in this article is a relatively simple and good option. However, please note that the contents explained here will change significantly if the software changes due to a major update of VS Code.

that's all