[PYTHON] Use OpenBLAS with numpy, scipy

If you feel that numpy and scipy are slow, you should check the blas settings. The environment I tried was ubuntu 14.04.

To check, launch python

>>> import numpy
>>> numpy.show_config()

lapack_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3']
    define_macros = [('NO_ATLAS_INFO', 3)]
blas_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    define_macros = [('NO_ATLAS_INFO', 3)]

Check at.

As a result of investigation, various methods such as using update-alternatives came out, but since that one had strange behavior, the method of directly setting OpenBLAS and compiling numpy is described as a memorandum.

reference

--Using BLAS on ubuntu 14.04 (link)

• Installing Numpy and OpenBLAS(link) -(numpy, scipy If you feel a little slow, check if blas, lapack, atlas are included link)

Advance preparation

Install python-dev and gfortran as preparations before installing OpenBLAS.

~$ sudo apt-get install git python-dev gfortran

OpenBLAS installation

Download OpenBLAS from git, make and install with gfortran. Make sure it is installed in / opt / openblas /.

~$ git clone https://github.com/xianyi/OpenBLAS
~$ cd OpenBLAS
~$ make FC=gfortran
~$ sudo make PREFIX=/opt/openblas install

After that, set the path to OpenBLAS. It is good to describe it in ~ / .bashrc etc.

~$ export LD_LIBRARY_PATH=/opt/OpenBLAS/lib:$LD_LIBRARY_PATH

When using with multiple users

~$ sudo -s "echo '/opt/OpenBLAS/lib' > /etc/ld.so.conf.d/openblas.conf"

After the setting is completed

~$ sudo ldconfig

Is executed.

numpy installation

Download numpy from git and describe openblas settings in site.cfg

~$ git clone https://github.com/numpy/numpy
~$ cd numpy
~$ cp site.cfg.example site.cfg
~$ vi site.cfg

Uncomment the following part of site.cfg.

[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include

After that, check if the setting change is successful with ~ $ python setup.py config.

openblas_info:
  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/openblas/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/openblas/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

After checking the above, build and install with ~ $ python setup.py build && python setup.py install.

Operation check

Check the speed with Code here. The number of threads to be used is specified by ʻOMP_NUM_THREADS` (the example below is a matrix calculation of 5000 * 5000).

~$ OMP_NUM_THREADS=1 python test_numpy.py
version: 1.12.0.dev0+3c394f7
maxint:  9223372036854775807

BLAS info:
 * libraries ['openblas', 'openblas']
 * library_dirs ['/opt/openblas/lib']
 * define_macros [('HAVE_CBLAS', None)]
 * language c

dot: 4.537050 sec
~$ OMP_NUM_THREADS=8 python test_numpy.py
version: 1.12.0.dev0+3c394f7
maxint:  9223372036854775807

BLAS info:
 * libraries ['openblas', 'openblas']
 * library_dirs ['/opt/openblas/lib']
 * define_macros [('HAVE_CBLAS', None)]
 * language c

dot: 1.855611 sec

scipy installation

If you can install numpy above, all you have to do is install scipy with pip.

pip install scipy