[PYTHON] Pip dependency check becomes strict

Introduction

I also learned this by listening to the Python Bytes podcast, but it seems that the dependency check that pip performs when installing a module becomes stricter. If you don't care, one day you may suddenly be unable to install the dependent packages as soon as you upgrade pip. I will write about how it will change and what I can do to prepare for it.

pip's new dependency checker

I wasn't really aware of it, but with pip so far, I was able to install packages with inconsistent dependencies. pip's new dependency checker (2020-resolver) will no longer allow this.

For example, the virtualenv package (version 20.0.2) has the dependency six> = 1.12.0, <2. It says "six v1.12.0 or higher and less than v2", but you can intentionally try to install it in a way that breaks that dependency.

pip install "six<1.12" "virtualenv==20.0.2"

If you run this on pip v20.0.1 or earlier, it will look like this.

$ pip install "six<1.12" "virtualenv==20.0.2"
pip install "six<1.12" virtualenv==20.0.2
Collecting six<1.12
  Using cached six-1.11.0-py2.py3-none-any.whl (10 kB)
Collecting virtualenv==20.0.2
  Using cached virtualenv-20.0.2-py2.py3-none-any.whl (4.6 MB)
Collecting distlib<1,>=0.3.0
  Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting appdirs<2,>=1.4.3
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting filelock<4,>=3.0.0
  Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
ERROR: virtualenv 20.0.2 has requirement six<2,>=1.12.0, but you'll have six 1.11.0 which is incompatible.
Installing collected packages: six, distlib, appdirs, filelock, virtualenv
Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12 six-1.11.0 virtualenv-20.0.2

The message says ERROR, but the installation itself has been completed. You may miss it.

This will be the case from v20.0.2.

 pip install  "six<1.12" virtualenv==20.0.2
Collecting six<1.12
  Using cached six-1.11.0-py2.py3-none-any.whl (10 kB)
Collecting virtualenv==20.0.2
  Using cached virtualenv-20.0.2-py2.py3-none-any.whl (4.6 MB)
Collecting appdirs<2,>=1.4.3
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting distlib<1,>=0.3.0
  Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting filelock<4,>=3.0.0
  Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Installing collected packages: six, appdirs, distlib, filelock, virtualenv
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

virtualenv 20.0.2 requires six<2,>=1.12.0, but you'll have six 1.11.0 which is incompatible.

The error message has changed.

• Since the behavior of pip will change after October 2020, an error may occur when installing or upgrading packages.
• Run with the --use-feature = 2020-resolver option and try before the new resolver becomes the default.

So, let's run it with that option.

$ pip install --use-feature=2020-resolver "six<1.12" virtualenv==20.0.2
Collecting virtualenv==20.0.2
  Using cached virtualenv-20.0.2-py2.py3-none-any.whl (4.6 MB)
ERROR: Cannot install six<1.12 and virtualenv 20.0.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested six<1.12
    virtualenv 20.0.2 depends on six<2 and >=1.12.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

I can't install because I can't meet the dependencies. This will be the behavior of pip in the future.

Implementation schedule

The new dependency checker will be introduced on the following schedule.

What to do now

• Update pip to the latest 20.2.
• Use pip with --use-feature = 2020-resolver option

This allows you to anticipate the behavior of future versions. This will save you a lot of hassle in October, but it's especially useful if you're versioning using pip install -r requirements.txt or ʻinstall -c constraints.txt. This is because the requirements.txt or constraints.txt created by pip freeze` with previous versions of pip may have been written out with inconsistent dependencies, which you can check in advance. about it.

If you don't want to specify options one by one, go to ~ / .config / pip / pip.conf

[install]
use-feature=2020-resolver

If you write, it will be attached automatically. I have to remember to erase it when the version goes up in the future.

Summary

Since the behavior of pip's dependency check changes, I wrote about it. There are Poetry and pipenv for python package management, but the original pip is also It's evolving independently. I would like to write if there are any additional functions that interest me.