Introducing vebootstrap to handle virtualenv more easily in Python development environment

As of early 2018, Pipenv is recommended. I've noticed that the number of libraries that provide binaries for Windows has increased considerably, so (Mini | Ana) Conda shouldn't be a problem.

Speaking of Python development environment these days, I think the following patterns are the mainstream.

• Build Python development environment (pythonz, virtualenv, direnv)

Once you're ready, you'll have a flexible environment, but there are many steps and you'll need to deploy a lot of things other than python. Also, some are difficult to deploy on Windows. So, I made a pure python package that makes it easier to handle the virtualenv environment for each project.

Motivation

Originally, the ideal was that it was easy to set up, just set python hoge.py like normal script execution, the dedicated virtualenv environment was activated, and the necessary packages were included. The script was to launch. Simply put, the package that achieves this is the vebootstrap created this time.

vebootstrap https://pypi.python.org/pypi/vebootstrap https://github.com/tomoemon/vebootstrap

I don't really care about that because I don't switch versions of python that much when writing a little personal script (I should be able to use it in combination with pythonz etc. if necessary).

Operating environment

• OS
• OS that runs the following Python
• • Operation confirmed on Windows7 and CentOS6.5
• Python
• 2.7
• 3.3 or later
• • Operation confirmed with 2.7, 3.4, 3.5
• Dependent packages
• pip (Installed from the beginning in recent versions of python)
• virtualenv (installed automatically if needed when installing vebootstrap)

What you can do

• Separation of virtualenv environment for each project (folder)
• Automatically processes the following contents when executing a python script
• Create and enable virtualenv environment
• Installation of dependent packages using pip (automatic update when dependent packages are changed)
• Windows only, installation of packages such as numpy, lxml, Pillow (PIL) that cannot be installed by pip standard
• It does not depend on environment variables or the current directory of the shell (identify the dedicated virtualenv environment from the path of the script to be executed)

Things impossible

• Manage multiple versions of Python (use python as is, either in the path or explicitly specifying the full path)
• Reuse of virtualenv environment (premise to create a dedicated virtualenv environment for each project)

setup

Install vebootstrap using pip that corresponds to the python you plan to use. If you are using the system standard python in a Linux environment, you will need sudo.

$ pip install vebootstrap

How to use

Change the following directory names and script file names as appropriate. However, do not modify requirements.txt as vebootstrap will look for a file with that name. Also, in the sample, it is executed after cd in the project directory, but the same behavior can be obtained even if it is executed from outside the directory.

• Specifications are subject to change as it is under development.

** Simplest pattern **

Directory structure

project_test1/
    hoge.py

hoge.py

import vebootstrap

print("Hello vebootstrap")

Command execution

$ cd project_test1
$ python hoge.py

If you put ʻimport vebootstrap at the beginning of the main script hoge.pyand start it, it will be called.py ## (##is the python version) in theproject_test1directory at the first startup. Create a directory for your virtualenv environment. After that, executehoge.py with the virtualenv environment active, and activate and execute the existing .py ## `on the second and subsequent startups.

** If there are dependent packages **

Directory structure

project_test2/
    hoge.py
    requirements.txt

hoge.py

import vebootstrap
import requests

print(requests.get('http://yahoo.co.jp').text[:100])

requirements.txt

requests

Command execution

$ cd project_test2
$ python hoge.py

The same goes for creating a directory for the virtualenv environment the first time you start hoge.py, and use requirements.txt to pip install the dependent packages. Then activate the virtualenv environment and run hoge.py. If you rewrite requirements.txt, recreate the virtualenv environment at the next startup. (Since it may be difficult to uninstall individual packages, we use such specifications.)

** (Windows only) If you want to install a binary package for Windows **

For Windows users, installing packages that require their own build, such as numpy and lxml, can be a daunting task. You'll have to find and manually install the pre-built packages using something other than pip (PyPI), but vebootstrap makes this easy.

Specifically, I try to find the target package from the following site and install it. http://www.lfd.uci.edu/~gohlke/pythonlibs/

• This site is operated by an individual, so please use it at your own risk.

Directory structure

project_test3/
    hoge.py
    requirements.txt

hoge.py

import vebootstrap
import numpy

print(dir(numpy))

requirements.txt

#windows:numpy

Command execution

$ cd project_test3
$ python hoge.py

The usage is almost the same as before, but the description of requirements.txt is a little special. If there is a line of the form # windows: hogehoge, vebootstrap will look for the hogehoge package from the previous site. For pip, lines starting with # are treated as comments, so this is only interpreted by vebootstrap.

You can also install the packages that are installed by pip standard and the binaries for Windows as shown below.

requirements.txt

requests
#windows:numpy
beautifulsoup4

I don't know which package you have to install pre-built with this special writing! If you are, you should write hogehoge and requirements.txt as usual first, and if you get an unclear error, write # windows: hogehoge.

** If you don't want to include irrelevant import statements **

ʻImport vebootstrap` is the minimum magic to activate the virtualenv environment just by running python, but it may not be necessary if you are writing a script to expose to the outside world. In that case, please use the function to execute the module as a script as shown below. It behaves in the same way as when an import statement is inserted.

Directory structure

project_test4/
    hoge.py
    requirements.txt

hoge.py

import requests

print(requests.get('http://yahoo.co.jp').text[:100])

requirements.txt

requests

Command execution

$ cd project_test4
$ python -m vebootstrap hoge.py

** If it's a hassle to write dependent packages for most projects every time **

If you are a frequent project creator and write the same requirements.txt every time, try using the following functions.

Using vebootstrap even once creates a configuration file called .vebootstrap under your home directory (cd% USERPROFILE% for Windows users).

$HOME/.vebootstrap

{
    "default_packages": [
        "ipython"
    ], 
    "default_windows_packages": []
}

This file specifies the packages that should be installed in all projects regardless of the contents of requirements.txt. By default, the ʻipythonpackage is installed so that the REPL of the virtualenv environment can be used immediately. Add other packages as needed. Binaries for Windows can be installed automatically by adding them todefault_windows_packages in the form "numpy" `.

Example when adding some

$HOME/.vebootstrap

{
    "default_packages": [
        "ipython",
        "requests"
    ], 
    "default_windows_packages": [
        "numpy"
    ]
}

Notes

Note that the virtualenv environment is created for each version of python you run. When you type the following command, for example, if the version of python is 2.7.x, create a virtualenv environment with the name .py27 (the python executable file copied internally is 2.7.x. Things).

$ python hoge.py

If you execute the following command in the same directory, a virtualenv environment named .py35 will be created. (If the version of python3 is 3.5.x)

$ python3 hoge.py

at the end

Since my main environment is Windows, it may be difficult for Linux main people to use it. Please point out if you say "it cannot be used in such cases".

And although the package name vebootstrap is a bit long (and a little hard to type) personally, both ve and veb have already been taken ...