[Procedure memo] Install Python3 + OpenSSL locally on Ubuntu

Steps to install Python 3 locally while resolving OpenSSL errors without administrator privileges.

Table of contents

Motivation
  • Since the environment of the server with GPGPU in the laboratory to be used next time is old, the OS is old, python is old, and OpenSSL is also old, an OpenSSL related error occurred when using the globally installed pip or when installing python.
  • It is easier to move the environment if the files are arranged locally as much as possible.
  • There is also an option called Docker (nvidia-docker), but it seems that the environment construction work will increase further + Above all, it is annoying because an error occurred at the image creation stage on this server in the first place.

Environment / Conditions
  • Do not perform work that requires administrator privileges.
  • The home directory is / home / user (let's call it).
  • The installation destination is / home / user / local, and the source location is / home / user / src.
  • Installation target: Python3.7.9 (3.8.6 was fine.)
  • OS: Ubuntu14.04.6 LTS (old) 
$ cat /etc/issue
Ubuntu 14.04.6 LTS \n \l

Procedure

1. Download required files 
$ cd /home/user/src
$ wget https://www.openssl.org/source/openssl-1.1.1h.tar.gz
$ wget https://prdownloads.sourceforge.net/tcl/tcl8.6.10-src.tar.gz
$ wget https://prdownloads.sourceforge.net/tcl/tk8.6.10-src.tar.gz
$ wget https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz

2. Install OpenSSL

Related aliases

/home/user/.bash_profile


alias targz="tar xzvf"
alias tarxz="tar xvf"

Create installation destination
$ mkdir /home/user/local/ssl
$ mkdir /home/user/local/ssl/1_1_1h
Installation
$ cd /home/user/src
$ targz openssl-1.1.1h.tar.gz
$ cd openssl-1.1.1h
$ ./config --prefix=/home/user/local/ssl/1_1_1h --openssldir=/home/user/local/ssl shared
$ make
$ make install

Confirmation

/home/user/.bash_profile


Add local OpenSSL to linker search
LD_LIBRARY_PATH=/home/user/local/ssl/1_1_1h/lib:$LD_LIBRARY_PATH

alias /home/user/local/ssl/1_1_1h/bin/openssl

$ openssl version
OpenSSL 1.1.1h  22 Sep 2020
If this is displayed, it is successful (By the way, global openssl is 1.0.2）

3. Install tcl, tk ``` $ cd /home/user/src $ targz tcl8.6.10.tar.gz $ targz tk8.6.10.tar.gz $ cd tcl8.6.10 $ mkdir build $ cd build $ ../unix/configure --prefix=/home/user/local/ $ make $ make install $ cd tk8.6.10 $ mkdir build $ cd build $ ../unix/configure --prefix=/home/user/local/ $ make $ make install ```

4. Install python

Related aliases (functions) It was troublesome to write many times.

/home/user/.bash_profile


$ function pyconf (){
    ./configure --prefix=/home/user/local/$1 --with-ensurepip --with-openssl=/home/user/local/ssl/1_1_1h \
    --with-tcltk-includes="home/user/local/include" --with-tcltk-libs="/home/user/local/lib -ltcl8.6 -ltk8.6"
}

$ mkdir /home/user/local/python379
$ cd /home/user/src
$ pyconf python379
$ make
$ make install

5. Final confirmation

/home/user/.bash_profile


alias python37="$HOME/local/python379/bin/python3.7"
alias pip37="$HOME/local/python379/bin/pip3.7"

$ python37
Python 3.7.9 (default, Nov  2 2020, 16:54:48)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>>
OK if the ssl module can be imported normally
>>> exit()
$ pip37 install numpy
Also check if pip can be used

that's all.

Error collection

OpenSSL error when make install Python ``` Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host(). LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381 ``` The content of the message itself is that OpenSSL 1.0.2 to 1.1 is required as the text says, but this appears even though it seems that OpenSSL was installed as in Step 2 ( If you see it hundreds of lines before the end of the output when you do make install), you haven't found your local OpenSSL, so there's a mistake in --openssldir when you config, or "shared" (this). It is better to check if there is no problem (it has not been verified whether there is a problem with the presence or absence), and if the local OpenSSL is included in the search target of the linker.

SSL error during pip install 
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
Could not fetch URL https://pypi.org/simple/numpy/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/numpy/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement numpy (from versions: none)
ERROR: No matching distribution found for numpy
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

Well this is also due to OpenSSL not being able to link with Python. Review around the installation of OpenSSL. It should be okay if "openssl version" displays the version correctly, and for that matter the python interpreter can import the ssl module.

Reference / Related page

Someday I'll install all the other libraries locally as well. .. ..

Recommended Posts