[PYTHON] Using Intel MKL with NumPy / SciPy (November 2019 version)

Intel MKL is a high-speed numerical processing library developed by Intel that runs only on Intel CPUs. It includes linear algebra operations, FFT, etc. Numpy / SciPy uses openblas for linear algebra operations as standard. , Intel MKL is faster.

The content of this article has been validated on Debian 10 (native / WSL) using Python 3.8.0. * Officially Debian only supports 8 or 9 and Debian 10 is not. [On CentOS 7.7 Added the result. 2019-11-13]

Installing Intel MKL

download

Follow the links "Choose and Download", "Linux *", "Register & Download" from Intel MKL, and from the form your name and email address, etc. Fill in the required items and submit. As soon as you do, you will receive a download link by email (the attached file contains the key but did not use it). Open the link and select the one that suits your environment and download it.

There is also an option to install using apt or yum. However, I personally think that the standalone version is fine.

Installation

The tgz file will be downloaded, unzip it and move it into it. Then you will find two files in it, ./install.sh and ./silent.cfg. Simply sudo ./install. Running sh will start the interactive installation. Still, if you rewrite ./silent.cfg as appropriate and run sudo ./install.sh -s ./silent.cfg, the installation will be done automatically. Will go.

According to the article How to install mkl numpy, the registration key is required, but when I ran it, it was not. So you can edit ./silent.cfg only by rewriting ʻACCEPT_EULA = accept`, and don't create a field for the activation key.

Setting environment variables

Set the necessary environment variables around ~ / .profile or ~ / .bashrc.

export MKL_ROOT_DIR=/opt/intel/mkl
export LD_LIBRARY_PATH=$MKL_ROOT_DIR/lib/intel64:/opt/intel/lib/intel64_lin:$LD_LIBRARY_PATH
export LIBRARY_PATH=$MKL_ROOT_DIR/lib/intel64:$LIBRARY_PATH
export PKG_CONFIG_PATH=$MKL_ROOT_DIR/bin/pkgconfig:$PKG_CONFIG_PATH

PKG_CONFIG_PATH is not needed for the current purpose, but it is useful if you want to use Intel MKL other than Python, so set it at the same time.

Install NumPy and SciPy

Build

First, create a file called ~ / .numpy-site.cfg and write the following contents.

:.numpy-site.cfg


[mkl]
library_dirs = /opt/intel/mkl/lib/intel64/
include_dirs = /opt/intel/mkl/include
mkl_libs = mkl_rt
lapack_libs =

Then install with pip. To force compilation instead of downloading the binary package

$ pip install --no-binary :all: numpy
$ pip install --no-binary :all: scipy

It takes a long time to build (especially SciPy, about 12 minutes on the desktop), so you have to wait patiently.

__ [Addition 2019-11-13] __ When I ran it on CentOS 7.7, it was as old as gcc 4.8, and I had to enable C99 mode. You can flag it as follows.

$ CFLAGS="-std=c99" pip3 install --no-binary :all: numpy
$ CFLAGS="-std=c99" pip3 install --no-binary :all: scipy

Operation check

$ python3.8
iPython 3.8.0 (default, Oct 15 2019, 23:02:58)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import numpy as np
>>> np.show_config()
blas_mkl_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
blas_opt_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
lapack_mkl_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
lapack_opt_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
>>> 
>>> from scipy import show_config
>>> show_config()
lapack_mkl_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
lapack_opt_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
blas_mkl_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
blas_opt_info:
libraries = ['mkl_rt', 'pthread']
library_dirs = ['/opt/intel/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['/opt/intel/mkl/include']
>>> 

References

-How to install mkl numpy May 2016 -Install Intel MKL from apt October 2018 -Using MKL with numpy October 2018

Recommended Posts

Using Intel MKL with NumPy / SciPy (November 2019 version)
Use OpenBLAS with numpy, scipy
Harmonic mean with Python Harmonic mean (using SciPy)
Install NumPy, SciPy, Matplotlib with Homebrew on OS X (as of November 2015)
Use multithreaded BLAS / LAPACK with numpy / scipy
I want to use mkl with numpy and scipy under pyenv + poetry environment
Speeding up numerical calculation using NumPy / SciPy: Application 2
[Rust / Python] Handle numpy with PyO3 (August 2020 version)
Speeding up numerical calculation using NumPy / SciPy: Application 1
LPC with Scipy
Try using scipy
ICA with Scipy
CORDIC with Scipy
FFT processing with numpy and scipy and low pass filter