[PYTHON] About package management with conda and pip

In a word, Anaconda comes with a library of scientific calculations in addition to Python itself from the beginning. It has also been devised to simplify package management and deployment and comes with a package management command called conda.

About conda and pip

The Python world has long come with its own package management tools, which used to be EasyInstall, and you'll need to use pip later. I had to set it up. It was troublesome one by one. Python nowadays comes with pip by default, which is the best time. With this command, you can install the package published on the site called PyPI via the Internet with the command. It's the best time.

Now, a common story is the discourse that "breaks" when conda and pip are used together. I'm not sure what it claims to break, but it's unusual to put it in the same package with conda or pip in the first place. It's not the package management system that's broken, but rather your head trying to do something incoherent.

Don't get me wrong, Anaconda puts the package in by typing the basic conda command, but you can also use pip. The former conda is entered from the Anaconda repository. The latter pip comes from PyPI. Of course, the latter has more packages. Also, PyPI doesn't have a review, so anyone can be a package author. In the extreme, even if you write strange source code and publish it on PyPI, no one has checked it. If you package source code that you only use for your project and put it on your PyPI, no one will tell you.

On the other hand, Anaconda's repository is managed by the company, so I think I'm checking it as it is, but I'm not a person inside, so I'm not going to guarantee "I'm checking!".

There is also a community-run repository called conda-forge, which you can also enter with conda. But at the time of using it, it seems that it is a product of the community after all.

To put it bluntly, the dependencies between packages depend on the author's convenience, so no one looks around the entire Python world and rigorously examines those relationships. It depends on you who use it.

This kind of discussion has been around for a long time not only in Python but also in Ruby's package management system, for example, and after all, everyone makes packages as they like, so Freedom. Open source is such a thing. I think the atmosphere is such that the user should verify it.

I personally think that it is only Debian stable version that thoroughly verifies the dependencies between packages and maintains the quality.

Displaying dependency information in pip

As the package maintainer and author know, when registering with PyPI, write various meta information in the package distribution file.

So, how to check the dependency information can be displayed with the following command.

pip install pipdeptree #Put in with pip
pipdeptree -p <package name>

For example, here are some examples of packages called pandas for each version.

pandas==0.16.2
  - numpy [required: >=1.7.0, installed: 1.11.2]
  - python-dateutil [required: >=2, installed: 2.6.0]
    - six [required: >=1.5, installed: 1.10.0]
  - pytz [required: >=2011k, installed: 2016.7]

pandas==0.22.0
  - numpy [required: >=1.9.0, installed: 1.13.3]
  - python-dateutil [required: >=2, installed: 2.6.1]
  - six [required: >=1.5, installed: 1.11.0]
  - pytz [required: >=2011k, installed: 2017.2]

This will give you an idea of which package depends on which version of which package.

Show duplicate packages in both conda and pip

A common accident is that you manage it with conda, but you unintentionally put it in with pip. So duplicate packages can be viewed with the one-liner below.

conda list | cut -d ' ' -f 1 | sort | uniq -d

It's a good idea to check occasionally to make sure there are no duplicates.

Summary

That's why it became longer, but in summary

-Anaconda's package management comes with the conda and pip commands. -Basically, use conda to refer to the Anaconda repository. ・ If it is not in conda, use pip to enter it. -Occasionally check for duplicates, and if there are duplicates, give priority to conda.

You should build the target environment with reference to these.

I think it looks like this. I admit any objections, so I would appreciate your opinions. It's rough, but that's it.

Recommended Posts

About package management with conda and pip
Python installation and package management with pip
Notes on package management with conda
[Package management] Installation destination and internal processing of apt and pip
Install Python 2.7.9 and Python 3.4.x with pip.
About pip
About _ and __
Update package with pip standard features only
Package management in Maya-try putting pip in mayapy
Install Python package management tool pip (Windows)
I can't install the package with pip.
How about creating a virtual environment with Anaconda and doing pip install?
What is the difference between `pip` and` conda`?
Notes about with
mod_wsgi talks about putting it in with pip
Install pip and pandas with Ubuntu or VScode
conda vs. pip
pip vs conda
Getting Started with python3 # 2 Learn about types and variables
Let's make dependency management with pip a little easier
Start numerical calculation in Python (with Homebrew and pip)
python package dependencies and virtual environment management tool Poetry
Dealing with pip and related installation errors on Ubuntu 18.04
Install tweepy with pip and use it for API 1.1
About Class and Instance
PIL installation with pip
[Python] pip and wheel
Installing packages with pip3
Error with pip install
About cumprod and cummax
Use pip with MSYS2
Routine management with calendar
About cross-validation and F-number
With and without WSGI
Anaconda Package Management Notes
Install scikit.learn with pip
About "Lamvery", a deployment and management tool for AWS Lambda
Easily create authentication, user management, and multilingual systems with Flask-AppBuilder
Can't find the package you installed with pip install --user?
I tried follow management with Twitter API and Python (easy)
Steps to install a Git cloned package locally with pip
Move what you installed with pip to the conda environment
A story about installing matplotlib using pip with an error
Package python runtime and pypi library with chef / omnibus and Docker