At this time, I tried various Python virtual environments such as Anaconda and pyenv.
This is the article on the 18th day of NTT DoCoMo R & D Advent Calendar 2020.
Introduction
Hello. I'm Ishikawa, a DoCoMo employee.
I'm always doing data analysis, especially machine learning, and I often play around with Python, but when I noticed it, the virtual environment I was using was messed up, so I took this opportunity to organize it once. I thought I'd try it.
By the way, my environment before organizing is that pyenv is installed first, and anaconda3.6 is set up in that one virtual environment, that is, it feels like the virtual environment is inside the virtual environment.
Moreover, there weren't many changes in the environment, and despite this intricate construction, it didn't make sense at all.
Well, it works without any problems, so I thought it would be okay as it is, but "** Maybe the operation is uselessly heavy? " " I want to use a new Python version, but I want to set up a new environment I thought, "Is it annoying? **", so I decided to sort it out.
So, this time, I will try various virtual environments and summarize what kind of features each has. I hope it will be helpful for you to build a better Python development environment.
The execution environment in this article is Ubuntu: 20.04 running on Docker.
When referring to the execution command in the sentence, execute it with sudo as appropriate.
Also, at the end of the article, I summarized "This environment is recommended for such people", so I think you may look at that first.
This time, I referred to this article "pyenv, pyenv-virtualenv, venv, Anaconda, Pipenv. I use Pipenv.".
Virtual environment type
I tried the following 7 types.
- pyenv
- virtualenv
- pyenv-virtualenv
- venv
- Anaconda --Plain Python
- Pipenv
pyenv
What you can do with pyenv
--Installation of each Python version --Switching Python versions --Automatic switching of versions used under a specific directory
... apparently ...
Introduction of pyenv
apt install -y \
build-essential \
libffi-dev \
libssl-dev \
zlib1g-dev \
liblzma-dev \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
git \
wget
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
source ~/.bashrc
List of Pythons you can install
pyenv install --list
bash:Python3.9.Stall 0
pyenv install 3.9.0
bash:Default Python 3.9.Specify 0
pyenv global 3.9.0
Run python
python
numpy installation
pip install numpy
Reference: The story of installing pyenv on ubuntu 20.04 / 18.04
Impressions of pyenv
Advantages
--Very simple. --You can easily install and switch new versions of Python. ――Since pip is installed without permission, you can easily install Python.
Disadvantages
――Even though you have rewritten .bashrc, it doesn't have that much functionality, so some people may like it or not.
--The package environment cannot be separated or switched within the same Python version.
virtualenv
What you can do with virtualenv
--Switching between Python version and package environment
Will be. Installation of Python itself is required separately.
Introduction of virtualenv
apt install virtualenv
Creating a virtual environment
#To any place'test_env'Create an environment storage location with the name of
mkdir ~/test_env
#Designated as the storage location for the virtual environment
virtualenv ~/test_env #In this case the default Python version of the machine will be used
virtualenv -p /usr/bin/python3.9 ~/test_env #When you want to specify the version, specify the installed one like this
Start virtual environment
. ~/test_env/bin/activate
Run python
python
numpy installation
pip install numpy
Stop virtual environment
deactivate
Delete virtual environment
rm ~/test_env -r
Reference: How to install and configure Virtualenv (Ubuntu)
Impressions of virtualenv
Advantages
--You can switch the package environment within the same Python version. ――It feels lighter than other package management
Disadvantages
--Cannot install new Python using virtualenv. --It is troublesome because you have to type commands each time you start, stop, and switch the virtual environment.
pyenv-virtualenv
In the first place, pyenv-virtualenv seems to be a plugin of pyenv.
As a function, in addition to that of pyenv
--Switching between Python version and package environment --Automatic switching between Python version and package environment used under a specific directory
... apparently ...
pyenv-Introduction of virtualenv
#With pyenv installed
git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv
eval "$(pyenv virtualenv-init -)" >> ~/.bashrc
source ~/.bashrc
Creating a virtual environment
# Python3.9.At 0'test_env'Create an environment with the name of (Python3 with pyenv).9.0 is pre-installed)
pyenv virtualenv 3.9.0 test_env
Start virtual environment
pyenv global test_env
Run python
python
numpy installation
pip install numpy
Stop virtual environment
source deactivate
Delete virtual environment
pyenv uninstall test_env
Impressions of pyenv-virtualenv
Advantages
--You can easily install a new version of Python. --You can switch the package environment within the same Python version. --Simple creation of virtual environment. --You have to type commands to start, stop, and switch the virtual environment, but like pyenv, you can assign a specific virtual environment to a specific directory and switch automatically.
Disadvantages
--When executing all the commands, the environment check is executed once at the back, so the operation becomes heavy.
--It is necessary to rewrite .bashrc quite a bit.
venv
The characteristics of venv were almost the same as virtualenv.
However, the method of installing each Python version is synonymous with installing venv in the first place, so it feels simpler.
Introduction of venv
#This time 3.Create a virtual environment in 9.
apt install python3.9-venv python3-pip
Creating a virtual environment
# 'test_env'Create an environment with the name of
python3.9 -m venv ~/test_env
Start virtual environment
source ~/test_env/bin/activate
Run python
python
numpy installation
pip install numpy
Stop virtual environment
deactivate
Delete virtual environment
rm ~/test_env -r
Reference: Install Python 3.8 and venv virtual environment on Ubuntu
Impressions of venv
--Almost the same as virtualenv. ――It is officially provided, so you can use it with confidence.
Anaconda
What you can do with Anaconda
--Installation of each Python version
--Managing the package environment
--Package installation (can be done with the conda command instead of pip)
... apparently ...
Miniconda(Lightweight version of Anaconda)Introduction of
cd ~
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod 777 Miniconda3-latest-Linux-x86_64.sh
~/Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
Creating a virtual environment
# Python3.8.0'test_Create a virtual environment with the name env ’
conda create -n test_env python=3.8.0
Start virtual environment
conda activate test_env
Run python
python
numpy installation
#You can do it with the conda command.
#You can also use the pip command, but it seems better to use either one so that it does not collide.
conda install numpy
Stop virtual environment
conda deactivate
Delete virtual environment
conda remove -n test_env --all
Reference: Installing miniconda and duplicating / importing virtual environment
Impressions of Anaconda
Advantages
--Beginner-friendly because it installs Python, manages the package environment, and installs packages. ――Anaconda takes care of everything, so it's easy to manage.
Disadvantages
--Various saved in your home directory.
--It is troublesome because you have to type commands each time you start, stop, and switch the virtual environment.
--The conda command feels heavy.
-Some usage restrictions apply
Plain python
What is "plain Python" here?
--Install each Python version with apt install
--Do not manage the package environment
--Python version is specified by command at runtime
This is a very primitive method.
Python3.9 installation
apt install python3.9
apt install python3-pip
pip3 install -U pip
Run python
python3.9
numpy installation
python3.9 -m pip install numpy
Impressions of plain Python
Advantages
--Minimal installation. The lightest.
Disadvantages
--Slightly complicated installation
――Of course, you cannot manage the package environment.
--Execution commands and pip commands are complicated. However, if you specify alias as shown below, you can execute it with the python or pip command like any other command.
echo "alias python=\"python3.9\"" >> ~/.bashrc
echo "alias pip=\"python3.9 -m pip\"" >> ~/.bashrc
source ~/.bashrc
Pipenv
What is Pipenv?
--Independent of Python itself, of course there is no Python installation feature
--You can install packages instead of pip and manage the package environment at the same time.
That is.
Installing Pipenv
apt install python3-pip python3-distutils python3-dev
pip3 install --upgrade setuptools
pip3 install pipenv
echo "export PIPENV_VENV_IN_PROJECT=true" >> ~/.bashrc
source ~/.bashrc
Creating a virtual environment
#Python3 with apt.Install 9 and ’~/test_env'Python 3.Create 9 virtual environments
apt install python3.9
mkdir ~/test_env
cd ~/test_env
pipenv --python 3.9
#If you install pyenv, it will install it if you have not installed it at this time.
numpy installation(When the virtual environment is not started)
pipenv install numpy
Launch virtual environment
pipenv shell
numpy installation(When the virtual environment is booting)
pipenv install numpy
#Or
pip install numpy
Python execution
python
Stop virtual environment
exit
Delete virtual environment
pipenv --rm
Reference: [Ubuntu 18.04] Build Python environment with pyenv + pipenv, Install pipenv and moviepy on Ubuntu 20.04
Impressions of Pipenv
Advantages
--You can easily see the list of installed packages and their versions for each virtual environment.
~/test_env/Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
numpy = "*"
[dev-packages]
[requires]
python_version = "3.9"
Disadvantages
--The pipenv install command is heavy.
--Cumbersome to create, start and stop virtual environment.
General comment (Let's use this virtual environment for such people)
I've tried 7 types of virtual environments this time, but is this suitable for people who say this after actually using it? I tried to summarize what I thought.
I hope that it will be useful for those who are building an environment or who want to review it.
-
** No need for package management ** Rarely change the ** version of Python **, I'm used to installing with apt, etc.
-
Plain python
-
** No need for package management ** but Python ** I want to switch versions **, I want to update. Anyway ** I want to build a Python environment quickly, simply and easily **.
- pyenv
-
** Package management required **, ** not troublesome ** I want to switch
- pyenv + pyenv-virtualenv
** ** Package management required ** and I don't want the commands to be ** heavy ** * venv
-
Co-coding ** Everyone is using Pipenv **, or ** I want to make package management smarter **
- pyenv + Pipenv
-
Other than the above, or ** I want you to put everything around Python together **
- Anaconda
Reflection and commandments
As an aside from here, I would like to write about what I learned from this writing.
During this research, I had an idea, "** I'm using Dokcer in the first place, so I don't need a Python virtual environment **". It feels like I was just installing it in a container without having to use pyenv or Anaconda. Despite the wonderful features and merits of each of pyenv and Anaconda, it was a bitter situation to say that they were slowing down without using them at all.
Do you work or research while understanding not only the Python environment but also what you use all the time and the characteristics of the environment you are in? You may be able to maximize your performance by stopping once, facing them head-on, and capturing their original characteristics.
We hope that this article will improve the work and research activities of as many people as possible and the Python environment.
Recommended Posts