I listened to the PythonBytes podcast Latest Episode (208th) and introduced some interesting tools, so I tried it. I think that dependency management with pip will be a little easier.
First, a basic review of dependency management with pip. Use pip install to install the required packages. For example, if you install two packages, requests and pandas, you will get the following.
$ pip install requests
Collecting requests
Downloading requests-2.25.0-py2.py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 7.2 MB/s
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
|████████████████████████████████| 136 kB 10.5 MB/s
Collecting certifi>=2017.4.17
Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)
|████████████████████████████████| 155 kB 12.9 MB/s
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Installing collected packages: urllib3, certifi, chardet, idna, requests
Successfully installed certifi-2020.11.8 chardet-3.0.4 idna-2.10 requests-2.25.0 urllib3-1.26.2
$ pip install pandas
Collecting pandas
Downloading pandas-1.1.4-cp39-cp39-macosx_10_9_x86_64.whl (10.3 MB)
|████████████████████████████████| 10.3 MB 5.9 MB/s
Collecting numpy>=1.15.4
Using cached numpy-1.19.4-cp39-cp39-macosx_10_9_x86_64.whl (15.4 MB)
Collecting pytz>=2017.2
Downloading pytz-2020.4-py2.py3-none-any.whl (509 kB)
|████████████████████████████████| 509 kB 31.7 MB/s
Collecting python-dateutil>=2.7.3
Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
|████████████████████████████████| 227 kB 33.8 MB/s
Collecting six>=1.5
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: numpy, pytz, six, python-dateutil, pandas
Successfully installed numpy-1.19.4 pandas-1.1.4 python-dateutil-2.8.1 pytz-2020.4 six-1.15.0
If you look at this, it will automatically pull not only the specified package but also the packages on which it depends. Use pip freeze if you want to see a list of installed packages.
$ pip freeze
certifi==2020.11.8
chardet==3.0.4
idna==2.10
numpy==1.19.4
pandas==1.1.4
python-dateutil==2.8.1
pytz==2020.4
requests==2.25.0
six==1.15.0
urllib3==1.26.2
This isn't something that Python has decided on, but many people write this result to a file called requirements.txt and then batch it up when they're reinstalled or used by others. You can install it with.
$ pip freeze > requirements.txt
$ pip install -r requirements.txt
The result of pip freeze is not only the packages that pip install, but also the dependent packages are listed as well. In addition, the version of each package is also included, and you can create the exact same environment as when you installed it by pip install -r. I think that many other tools call it a lock file, but in the case of pip, it is called "freeze".
It's nice that all the packages are listed in the sense that it reproduces the same environment, but if you look at this, you can't tell which one you installed and which one was pulled by the dependency. When uninstalling a package, you can delete what you package install, but it is difficult to delete it unless you know which package it pulled. Another problem is that you will continue to use the older version even if the dependent packages have bug fixes. Therefore, it is a tool that will be introduced in the next chapter.
pip-chill
There is a tool called pip-chill as a tool to make up for the inconvenience of pip freeze. Unlike pip freeze, it is not built into pip and must be installed separately.
$ pip install pip-chill
You can now use pip-chill. If you run it with no arguments, it will look like this.
$ pip-chill
pandas==1.1.4
pip-chill==1.0.0
requests==2.25.0
Similar to pip freeze, but only shows the packages you installed. It's convenient because you can tell at a glance "What did you install?"
The fact that pip-chill is on this list is a flaw in the ball, and even in the podcast, Brian said," I wish I had the option to hide pip-chill itself from the list. " , pip-chill | grep -v pip-chill can be avoided for the time being.
pip-chill has several options, for example --no-version will only show the package name.
$ pip-chill --no-version
pandas
pip-chill
requests
For example, at the stage of development, I don't think it is necessary to specify the version in detail, so I think it is possible to put this in requirements.txt. You can easily pull it out when you don't need it on the way, or try multiple packages with similar functions. So, after the test, it ’s okay! If it becomes, then pip freeze> requirements.txt and fix it including the dependent packages.
There is another useful option that you can use to get this output.
$ pip-chill -v
pandas==1.1.4
pip-chill==1.0.0
requests==2.25.0
# certifi==2020.11.8 # Installed as dependency for requests
# chardet==3.0.4 # Installed as dependency for requests
# idna==2.10 # Installed as dependency for requests
# numpy==1.19.4 # Installed as dependency for pandas
# python-dateutil==2.8.1 # Installed as dependency for pandas
# pytz==2020.4 # Installed as dependency for pandas
# six==1.15.0 # Installed as dependency for python-dateutil
# urllib3==1.26.2 # Installed as dependency for requests
After the normal output, a list of packages installed as dependencies is lined up in the form of commented out, and it gives information about "which package is dependent on". No, I wanted something like this.
In addition, -a will show you all the packages (that is, the same as pip freeze).
In addition, pip-chill does not remember and process the history of pip install, but displays" the one that no one has a dependency on (that is, the one that would have been installed manually) ". I'm just doing it. It's good to realize what you want to do with a simple mechanism. However, because of such a mechanism, for example, if you uninstall pandas from the above state, it will be like this.
$ pip uninstall pandas
$ pip-chill
numpy==1.19.4
pip-chill==1.0.0
python-dateutil==2.8.1
pytz==2020.4
requests==2.25.0
Obviously, pandas is gone, so numpy, python-dateutil, and pytz, which have been dependent on it, are listed above. It would be even more convenient if you could erase these together when you erased pandas.
In addition, "Chill" means "to cool". In Japanese, the refrigerator has a function called "chilled", which is the etymology (?). I don't go to Freeze, but I think it's a good name because it feels like it's going to cool down before that.
I tried using a tool called pip-chill. It does what I was wondering if pip freeze doesn't have this feature. I wonder if it will be imported into pip someday ...
Recommended Posts