Python virtual environment construction (2017 version) pyenv and pyenv-virtualenv and virtualenv and virtualenv wrapper and pyvenv and venv

Postscript 2017/01/29

I had you write a plunge article. Thank you very much.

As @shibukawa mentioned in the comment section, virtualenv and venv are almost the same, so there are actually two choices, pyenv or virtualenv (venv), but virtualenv has already established itself as the de facto standard.

I recommend pyenv in the article, but I will use virtualenv. I'm a light Python user, so I want to make the surrounding environment as de facto as possible and enjoy it.

Introduction

If you're looking into building a Python environment, you'll probably see the tool names listed in the title. The point is that it is an environment isolation tool, but these tools change rapidly, and beginners do not know what it is. I'm not so heavy with Python myself, so I've summarized the features and usage of each.

Do you really need that virtual environment?

Python can be used for full-scale development, but it is also very useful as a jig tool to streamline daily tasks. In the introductory article, it is taken for granted to use these environment isolation tools, but I think it is a bad habit just to raise the threshold unnecessarily. You don't need these tools while using them as your own convenience tools. Let's enjoy programming with Python rather than being exhausted by creating an environment.

testing environment

    $ lsb_release -a
    ...
    Description:    Ubuntu 16.04.1 LTS
    ...

    % python3 --version
    Python 3.5.2

    % pip3 --version
    pip 9.0.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)

    $ pip3 freeze | grep virtualenv
    virtualenv==15.1.0
    virtualenvwrapper==4.7.2

    $ pyenv --version
    pyenv 1.0.7-1-g99d1670

Relevance of each tool

There are six, so I don't know the translation immediately, but as shown in the list below, there are two sets, so you can think that there are actually three types.

pyenv-virtualenv is a plugin for pyenv. Is virtualenvwapper a wrapper script for virtualenv? , pyvenv is also treated as a wrapper for venv. Note that some articles confuse pyenv-virtualenv with virtualenv. The names are too confusing, but the two are different.

Try pyenv

Use this when you want to use multiple different versions of Python. Even though there are multiple versions, it is not a big group of Python 2 series 3 series, It is possible to manage fine version divisions such as 3.3 and 3.4, 3.3.0 and 3.3.1.

Managing Python for each version is not a difficult task because you only have to install it separately for each version and replace the symbolic links in a timely manner, but specialized tools will centrally manage it. Is still convenient.

With pyenv, it's easy to give it a try as a new version of Python has been released.

Unfortunately, as of 2017/01, it does not seem to be compatible with Windows. It seems that it is developed with shell script main as much as possible, so it seems unlikely that we can expect support in the future.

    #A list of installable versions is displayed. Not just regular Python
    #Python distributions such as anaconda and jython,Another like pypy
    #I'm surprised that even the Python implementation is listed.
    $ pyenv install --list

    #Install by specifying the version
    $ pyenv install 3.4.3
    $ pyenv install 3.6.0

    #If you specify a fluffy version, the corresponding version will be displayed.
    #It is convenient because it will list it.
    $ pyenv install 3.4

    python-build: definition not found: 3.4
    The following versions contain '3.4' in the name:
      3.3.4
      3.4.0
      3.4-dev

    #It seems that you need to execute this command after a new installation.
    #I hope it will be done automatically.
    $ pyenv rehash

    #Specifying the Python version to use globally
    $ pyenv global 3.4.0
    $ pyenv version
    Python 3.4.0

    #Specifying the Python version to use locally in the directory
    $ cd ~/project
    $ pyenv local 3.3.4
    $ pyenv version
    Python 3.3.4
    $ cd ~
    $ pyenv version
    Python 3.4.0

    #Specifying the Python version to use shell-locally
    $ pyenv global 3.4.0
    $ pyenv version
    Python 3.4.0
    $ bash
    $ pyenv shell 3.3.4
    $ pyenv version
    Python 3.3.4
    $ exit
    $ pyenv version
    Python 3.4.0

one point As for how to realize the function of directory-local Python, nothing great is done and the basic idea is very simple. global The specified Python version is saved as a string in the text file ~ / .pyenv / version. If you specify a directory local, a text file called .python-version is created directly under the directory, and the version string is saved there. Each version of Python is saved in ~ / .pyenv / versions for each version, and when switching versions, only a symbolic link to the corresponding Python is replaced. It's the first idea that comes to mind when you try to achieve the same thing yourself. It's a digression that it's a waste to just use it as a complete black box, as widely used tools don't always use special technology.

Try pyenv-virtualenv

virtualenv and pyenv-virtualenv are separate applications. Please note that some articles may be confused. pyenv-virtualenv is a plugin for pyenv. You can install multiple Python versions with pyenv, but the directory site-packages that manages the modules for each Python version is common. You can use pyenv-virtualenv to manage different site-packages in the same Python version. You can minimize the difference between the development environment and the execution environment by fixing all the module versions used in the application under development or installing only the minimum required modules.

    # pyenv-Install virtualenv.
    $ git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

    # .Add initial settings to xxxrc
    #Why you are using eval, optional`-`I was wondering what it was, so I asked a question on Stack Overflow.
    #  http://ja.stackoverflow.com/questions/32043/xxxenv-Eval at initialization of-xxxenv-init-Meaning of
    vi ~/.bashrc
        eval "$(pyenv virtualenv-init -)"

    #Arbitrary name for pyenv virtualenv to identify the Python version and virtualized environment(tag)Is specified.
    $ pyenv virtualenv <pyversion> <tag>

    #Then the corresponding site-The packages directory will be generated
    Requirement already satisfied: setuptools in /home/vagrant/.pyenv/versions/<pyversion>/envs/<tag>/lib/python<X.Y>/site-packages
    Requirement already satisfied: pip in /home/vagrant/.pyenv/versions/<pyversion>/envs/<tag>/lib/python<X.Y>/site-packages

    #You can check the created virtual environment from versions.
    $ pyenv versions
      system
      3.5.3/envs/virtual-3.5.3 # <<The two are probably the same
      virtual-3.5.3            # << 

    #Specifying the created tag name in the activate subcommand enables the virtual environment.
    $ pyenv activate <tag>

    #The tag name should be displayed to the left of the shell prompt like this
    (<tag>)$
  
    #Run deactivate to terminate the virtual environment.
    $ pyenv deactivate

    #You can create a directory-local virtualenv just like a regular pyenv.
    #Add pyenv local to the root directory of your application development project
    #It is a good idea to specify it.
    $ pyenv local <tag>

virtualenv and virtualenvwapper

I don't know much about the history of Python, but it seems that virtualenv was originally famous as a virtual environment construction tool, and pyenv seems to be a latecomer. If you try virtualenv after using pyenv, it certainly feels less familiar than pyenv. ~~ virtualenv cannot be used to switch the version of Python itself ~~ (I was pointed out in the comment. It seems that it can be switched), and also in terms of isolation of site-packages, pyenv-virtualenv I felt that it was easier to use. It may be natural because pyenv, pyenv-virtualenv is a latecomer, but there is a sense of disappointment. Since the history is old, there seems to be a long day about the surrounding environment such as editor plug-in support.

I may not use it anymore for creating a new environment, but I tried all the functions for the time being.

Try using virtualenv alone

    #Install virtualenv
    $ sudo pip3 install virtualenv

    #Create a virtual environment by specifying a directory.
    $ python3 -m virtualenv <dir>

    #Then such a directory will be created. The directory name is virtualenv
    #It may vary depending on the version.
    # [Confirmation environment] Ubuntu 16.04 virtualenv 15.1.0
    $ cd <dir>
    $ ls
    bin  include  lib  pip-selfcheck.json

    #When you load the activate script of the generated directory with the source command
    #The virtual environment is enabled.
    $ source bin/activate

    #While the virtual environment is active, the root directory name of the virtual environment is displayed on the left side of the prompt.
    (<dir>)$

    #Deactivate the virtual environment with deactivate
    (<dir>)% deactivate

Try using virtualenvwapper

By the way, virtualenv alone cannot be easily deleted, and many directories for virtualenv are created, which is another inconvenience. By introducing virtualenvwapper, you can use utility commands to use virtualenv conveniently, which makes it a little easier to use. I installed it referring to Manage python environment with virtualenv.

    # virtualenv /Install virtualenvwrapper
    $ sudo pip3 install virtualenv
    $ sudo pip3 install virtualenvwrapper

    # ~/.Add the initialization code to xxxrc
    $ vim ~/.bashrc
        source /usr/local/bin/virtualenvwrapper.sh
        export WORKON_HOME=~/.virtualenvs

Let's log in again and use it.

    /usr/bin/python: No module named virtualenvwrapper
    virtualenvwrapper.sh: There was a problem running the initialization hooks.
  
    If Python could not import the module virtualenvwrapper.hook_loader,
    check that virtualenvwrapper has been installed for
    VIRTUALENVWRAPPER_PYTHON=/usr/bin/python and that PATH is
    set properly.

Oh, I got something that looks like an error message. Since I installed it with pip3, it was installed as python3 module, but since / usr / bin / python of Ubuntu 16.04 is python2.7, virtualenvwrapper is not installed. It seems that you are saying. It seems that you should specify the Python3 path in VIRTUALENVWRAPPER_PYTHON. How kind it is! I want to apprentice. Let's add the settings and log in again.

    $ vim ~/.bashrc
        export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3

This time I got another message.

    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/initialize
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/premkvirtualenv
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/postmkvirtualenv
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/prermvirtualenv
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/postrmvirtualenv
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/predeactivate
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/postdeactivate
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/preactivate
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/postactivate
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/get_env_details
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/premkproject
    virtualenvwrapper.user_scripts creating /home/vagrant/.virtualenvs/postmkproject

Looking at the contents, it's just a shell script. It seems that a hook script can be prepared when an event occurs. Well let's ignore it for now. I will use it normally for the time being.

    #When you run virtualenvwrapper, you'll see a bunch of available commands.
    $ virtualenvwrapper

    Commands available:
  
      add2virtualenv: add directory to the import path
  
      allvirtualenv: run a command in all virtualenvs
      ...

    #You can create a virtual environment with mkvirtualenv.--no-site-When package is not specified,
    # $VIRTUALENVWRAPPER_PYTHON site-It seems to take over the packages. Create a virtual environment
    #Given the purpose, it is usually--no-site-I think you should specify packages.
    $ mkvirtualenv --no-site-package <tag>
  
    #When created, the environment name will be added to the leftmost at the command prompt
    (<tag>)$

    #The body of the virtual environment with cdvirtualenv? Can move to the directory where
    $ cdvirtualenv
    $ pwd
    /home/vagrant/.virtualenvs/<tag>

    #When virtualenv is a single unit, the virtualenv directory that was placed directly under the virtual environment directory is
    # $WORKON_It seems to be placed under HOME. It was a little refreshing.
    $ ls
    bin  include  lib  pip-selfcheck.json

    #You can also delete virtual environments that could not be done with virtualenv alone.
    $ deactivate
    $ rmvirtualenv <tag>

pyvenv and venv

Let's support venv with features like virtualenv as standard! So it seems to be a standard module added from Python 3.3. There is no command called venv, so specify the module from python and use it.

Next, when I was researching pyvenv, I found such an article.

Since pyvenv is deprecated, talk about using venv

Also, I was wondering if it was something like pyenv and virtualenv ... but it was simpler. In conclusion, pyvenv is like syntactic sugar for venv, not completely different like pyenv and virtualenv.

    #How to call venv
    $ python3 -m venv <options>

    #The standard python library is Ubuntu 16.In 04, it is installed in the following directory.
    $ cd  /usr/lib/python3.5
    $ find `pwd` -name "*venv*"
    /usr/lib/python3.5/venv

    # 2.7 doesn't seem to have venv.
    $ cd  /usr/lib/python2.7
    $ find `pwd` -name "*venv*"

pyvenv can be used on Ubuntu 16.04 by installing the following packages.

    $ sudo apt-get install -y python3-venv

    #If you check with dpkg/usr/bin/pyenv looks like the main body.
    % dpkg -L python3-venv
    /.
    /usr
    /usr/share
    /usr/share/doc
    /usr/share/man
    /usr/share/man/man1
    /usr/bin
    /usr/share/doc/python3-venv
    /usr/share/man/man1/pyvenv.1.gz
    /usr/bin/pyvenv

Looking at the contents, it's really just calling venv. It is a mystery why I decided to create this. The reason why pyvenv is deprecated is that, as you can see in the reference link, calling it explicitly as a Python module with python -m is not a mess because the Python version is clear! It seems that. You can tell from the beginning! I want to say ... From now on, I will explain with python -m venv without using pyven v.

    # /usr/bin/pyvenv

    #! /usr/bin/python3.5
    if __name__ == '__main__':
        import sys
        rc = 1
        try:
            import venv
            venv.main()
            rc = 0
        except Exception as e:
            print('Error: %s' % e, file=sys.stderr)
        sys.exit(rc)

Try venv

    #Show help
    $ python3 -m venv -h

    #Creating a virtual environment
    $ python3 -m venv <dir>

    #Somehow a directory has been created. It's the same configuration as virtualenv.
    $ cd <dir>
    $ ls
    bin  include  lib  lib64  pyvenv.cfg  share

    #The usage is the same as virtualenv.
    $ source bin/activate
    (dir)$ 
    $ deactivate
    $

There is only a standard and it is quite simple. I'm happy that it's standard, but I felt that it wasn't enough for full-scale operation because it couldn't be used or had insufficient functions unless it was 3.3 or higher.


Summary

As of 2017/01, the combination of pyenv and pyenv-virtualenv is probably the most convenient. It's a pity that it doesn't support Windows, but for full-scale development, it seems better to use Linux on a virtual machine. Since virtualenv can be used on Windows, it seems that it can be used to build a clean environment when you want to check the operation of the distribution module. You can use venv, but it hurts that you can't use it with Python 2.7.

I think the simplest way to use Python as a jig tool is to refer to the information on this site.

Reference link

pyenv, pyenv-virtualenv

virtualenv, virtualenvwapper

pyvenv, venv

Recommended Posts

Python virtual environment construction (2017 version) pyenv and pyenv-virtualenv and virtualenv and virtualenv wrapper and pyvenv and venv
Python3 TensorFlow environment construction (Mac and pyenv virtualenv)
Environment construction with pyenv and pyenv-virtualenv
Build a virtual environment with pyenv and venv
python development environment -use of pyenv and virtualenv-
Python: Creating a virtual environment (venv), starting and stopping
Build a python virtual environment with virtualenv and virtualenvwrapper
Building and enabling a python virtual environment, etc. (venv)
Build a python virtual environment with virtualenv and virtualenvwrapper
[Python] Django environment construction (pyenv + pyenv-virtualenv + Anaconda) for macOS
Python environment construction and TensorFlow
venv: Python virtual environment management
python with pyenv and venv
python standard virtual environment venv
Environment construction of python and opencv
Virtual Environment Version Control Summary Python
Build python virtual environment with virtualenv
Python environment construction (pyenv, anaconda, tensorflow)
Python environment construction (pyenv + poetry + pipx)
Python3 environment construction with pyenv-virtualenv (CentOS 7.3)
Construction of Python local development environment Part 1 (pyenv, pyenv-virtualenv, pip installation)
[Ubuntu 18.04] Python environment construction with pyenv + pipenv
Python virtual environment and packages on Ubuntu
About the virtual environment of python version 3.7
Python3 + venv + VSCode + macOS development environment construction
Python and machine learning environment construction (macOS)
Build a python virtual environment with pyenv
Clean python environment with pythonz and virtualenv
How to create a Python virtual environment (venv)
Building a python environment with virtualenv and direnv
Python environment construction
Python environment construction (Anaconda + VSCode) @ Windows10 [January 2020 version]
Manage Python multiple version environment with Pythonz, virtualenv
Installation of Python3 and Flask [Environment construction summary]
Build a simple Python virtual environment without pyenv
python environment construction
[Venv] Create a python virtual environment on Ubuntu
Python --Environment construction
Work in a virtual environment with Python virtualenv.
Pillow environment construction --Virtual environment by virtualenv, interactive environment by iPython
Use jupyter-lab installed in python virtual environment (venv)
Python environment construction
[Django3] Environment construction and various settings summary [Python3]
Python 3.x environment construction by Pyenv (CentOS, Ubuntu)
From Python environment construction to virtual environment construction with anaconda
python environment construction
Virtual environment construction with Docker + Flask (Python) + Jupyter notebook
Building a Python environment for pyenv, pyenv-virtualenv, Anaconda (Miniconda)
Build a Python virtual environment using venv (Django + MySQL ①)
python package dependencies and virtual environment management tool Poetry
Install pyenv on Raspberry Pi and version control Python
Install Python environment on local PC (pyenv, venv on Mac)
Install pyenv and pyenv-virtualenv
python windows environment construction
homebrew python environment construction
Python development environment construction
python virtual environment Pipenv
Install python (pyenv, pyenv-virtualenv)
virtual environment in python
pyenv + fish environment construction
python2.7 development environment construction