[Understanding in the figure] Management of Python virtual environment by Pipenv

I didn't really understand how to use Pipenv, so it's a memo when I tried various movements while reading mainly the official document.

Since the relationship between each command is illustrated, I will write it in that flow. The numbers in the figure correspond to the paragraph numbers in this article.

Pipenv

Traditionally, Python virtual environments were often built with pip and verticalenv, but Pipenv is a tool that makes it easy to manage them all together.

--Internally using pip and verticalenv --Managing package dependencies so that builds always give the same results --Use files Pipfile and Pipfile.lock instead of requirements.txt --Close to bundler, composer, npm, cargo, yarn, etc.

Pipfile and Pipfile.lock

Pipenv will generate two files. Basically, you don't edit these directly, but I will suppress each role in order to understand the reason and the state of the virtual environment. Each file has the following roles and properties.

Pipfile --Manage various packages and their versions --Separate management of development packages --Manage your own scripts

Pipfile.lock --Manage the packages and their versions that the installed packages depend on --Hash management to protect against remote package tampering (mechanism introduced in pip version 8.0)

Let's take a look at each command and how to use it.

0. Preparation

Install Pipenv.

$ brew install pipenv

You can also install it with pip.

$ pip install pipenv

Initialize the virtual environment by specifying the Python version. If the specified version does not exist, it will be installed via pyenv.

$ pipenv --python 3.6

When the virtual environment is initialized, a file called Pipfile will be generated.

1. install

You can install the package with the install command.

Use existing Pipfile or Pipfile.lock

If there are Pipfile and Pipfile.lock files in the same hierarchy where the command is executed, the environment is built by referring to those files.

$ pipenv install

Use existing requirements.txt

You can build an environment managed by Piprnv from requirements.txt even if it is not originally managed by Pipenv.

$ pipenv install -r ./requirements.txt

Build a new environment

If there is no reference file, the environment will be built in the same way as pipenv --python <version>.

$ pipenv install --python 3.6

For the install command, a Pipfile.lock file is generated in addition to the Pipfile. Let's take a look at the contents.

Pipfile

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]             //Empty because I haven't installed the package yet

[requires]
python_version = "3.6" //Version 3.6 is specified

Pipfile.lock

{
    "_meta": {
        "hash": {
            "sha256": "415dfdcb118dd9bdfef17671cb7dcd78dbd69b6ae7d4f39e8b44e71d60ca72e7"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {}, //Empty because I haven't installed the package yet
    "develop": {} 
}

Add a package

I will add pandas as a trial.

$ pipenv install pandas

Then, each will be added as follows.

Pipfile

[packages]
pandas = "*" //No version specified

Pipfile.lock

    "default": {
        "numpy": {
            "hashes": [
                "sha256:03bbde29ac8fba860bb2c53a1525b3604a9b60417855ac3119d89868ec6041c3",
(Omitted)
                "sha256:f6a7421da632fc01e8a3ecd19c3f7350258d82501a646747664bae9c6a87c731"
            ],
            "version": "==1.18.0"
        },
        "pandas": {
            "hashes": [
                "sha256:00dff3a8e337f5ed7ad295d98a31821d3d0fe7792da82d78d7fd79b89c03ea9d",
(Omitted)
                "sha256:ee50c2142cdcf41995655d499a157d0a812fce55c97d9aad13bc1eef837ed36c"
            ],
            "index": "pypi",
            "version": "==0.25.3"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c",
                "sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a"
            ],
            "version": "==2.8.1"
        },
        "pytz": {
            "hashes": [
                "sha256:1c557d7d0e871de1f5ccd5833f60fb2550652da6be2693c1e02300743d21500d",
                "sha256:b02c06db6cf09c12dd25137e563b31700d3b80fcc4ad23abb7a315f2789819be"
            ],
            "version": "==2019.3"
        },
        "six": {
            "hashes": [
                "sha256:1f1b7d42e254082a9db6279deae68afb421ceba6158efa6131de7b3003ee93fd",
                "sha256:30f610279e8b2578cab6db20741130331735c781b56053c59c4076da27f06b66"
            ],
            "version": "==1.13.0"
        }
    },

It also specifies the packages and versions that pandas internally depends on, such as numpy and python-dateutil.

If you want to specify the version and install it, do as follows.

$ pipenv install pandas==0.15.0 //Specify a specific version

The version can be specified not only for a specific version, but also for a specified version or higher, excluding a specific version. For details, refer to Official Document.

Remove the package

Use the uninstall command to uninstall the installed package.

$ pipenv uninstall pandas

Then, both Pipfile and Pipfile.lock will be deleted as added by install.

• Pandas will be uninstalled from the virtual environment, but other packages that pandas depended on will remain. Use the clean command described below to uninstall these.

2. lock

Use the lock command to update the Pipfile.lock file from the contents of the Pipfile.

$ pipenv lock

In particular, if you do not specify the version when pipenv install, you will use this command when upgrading the package.

3. sync

The sync command is used to apply the contents of Pipfile.lock to the virtual environment.

$ pipenv sync

I think that this command is often used as a set with the lock command.

4. update

The update command locks and syncs at the same time.

$ pipenv update        //Do for all packages
$ pipenv update pandas //It is also possible to specify the package

Add the --dry-run option to see which packages can be updated (obsolete).

$ pipenv update --dry-run
✔ Success! 
//Old version (0.15.I put pandas of 0), so the new version (0).25.3) will tell you that it is available
Skipped Update of Package pandas: 0.15.0 installed, 0.25.3 available.

If you specify the version at pipenv install, the installation of the new version will be skipped even if you do not have the --dry-run option. In that case, specify an arbitrary version with pipenv install and perform overwrite installation.

5. run

Use the run command to execute a command in a virtual environment. Let's display the list of installed packages as a trial.

$ pipenv run pip list
Package         Version
--------------- -------
numpy           1.18.0 
pandas          0.15.0 
pip             19.3.1 
python-dateutil 2.8.1  
pytz            2019.3 
setuptools      43.0.0 
six             1.13.0 
wheel           0.33.6

Also, if you register a script in Pipfile, you can call the script with the run command.

Pipfile

[scripts]
list = "pip list"

$ pipenv run list //Same result as pipenv run pip list

6. shell

Use the shell command to enter the virtual environment.

$ pipenv shell

7. clean

If you have packages in your virtual environment that are not defined in Pipfile.lock, you can use the clean command to remove them from your virtual environment.

Try pipenv uninstall pandas after pipenv install pandas that came out earlier, and execute the clean command with the packages that pandas depends on remain in the virtual environment.

$ pipenv run pip list 
Package         Version
--------------- -------
numpy           1.18.0 //There are still packages that pandas depends on, such as numpy
pip             19.3.1
python-dateutil 2.8.1  
pytz            2019.3 
setuptools      43.0.0 
six             1.13.0 
wheel           0.33.6 

$ pipenv clean //Run clean
Uninstalling six…
Uninstalling pytz…
Uninstalling python-dateutil…
Uninstalling numpy…

$ pipenv run pip list //Check the result of clean
Package    Version
---------- -------
pip        19.3.1 
setuptools 43.0.0 
wheel      0.33.6 

You can see that the packages that pandas depended on are also firmly uninstalled.

8. check

The check command checks for security vulnerabilities to see if the current environment meets the PEP 508 requirements.

$ pipenv check

9. graph

If you want to see the package dependencies, use the graph command.

$ pipenv graph
pandas==0.15.0 //pandas is numpy, python-It turns out that it depends on dateutil and pytz
  - numpy [required: >=1.7.0, installed: 1.18.0]
  - python-dateutil [required: >=2, installed: 2.8.1]
    - six [required: >=1.5, installed: 1.13.0] // python-You can see that dateutil is even more dependent on six
  - pytz [required: >=2011k, installed: 2019.3]

Summary

I think this gives you a bird's eye view of Pipenv. It's persistent, but I'll post the related diagram at the beginning again.

Other alternatives

Pipenv looks like all the good things, but it seems to have some drawbacks such as spiciness when combined with docker and slow installation. As another alternative, I will ask the names of the following tools, so I will try to touch them soon.

• Poetry
• Pyflow
• pip-tools