[PYTHON] Build a flask app made with tensorflow and dlib to work on centos7

As the title says, we will build an environment where the flask application using tensorflow runs on ceontos 7.1. Introduction It is assumed that you have set up the environment of centos 7.1 with conoha like this article.

App configuration

I'm using python, flask, anaconda, tensorflow, dlib, opencv. The configuration of the app is as follows.

.
├── evaluation.py #Judge the image. Use dlib, opencv, tensorflow.
├── guniconf.py
├── lib #Where to put the trained data of dlib and tensorflow
├── model.py #Use a format called Model to make image data easier to handle
├── static
│   ├── css
│   │   └── style.css
│   ├── images/
│   ├── js
│   │   └── main.js
│   └── robots.txt
├── templates
│   ├── index.html
│   └── layout.html
└── web.py #Receives a request and returns the result

Insert anaconda

I will use pyenv to put anaconda. I referred to here for the installation of pyenv on centos. http://qiita.com/glostuan/items/6030e309542615470e0d

$ sudo yum install epel-release #Add epel repository
$ sudo yum install gcc zlib-devel bzip2 bzip2-devel readline readline-devel sqlite sqlite-devel openssl openssl-devel git
$ git clone https://github.com/yyuu/pyenv.git ~/.pyenv
$ vim .bash_profile
#to add
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

$ source .bash_profile #Reflect
$ pyenv install --list
# anaconda3-4.0.Check if there is 0
$ pyenv install anaconda3-4.0.0
$ pyenv global anaconda3-4.0.0
$ pyenv versions
  system
* anaconda3-4.0.0 (set by /home/develop/.pyenv/version)

Now you can use anaconda.

Installation of opencv

At first, I did a lot of research to install opencv, and I thought I would install the dependent libraries one by one, but the amount was so large that I couldn't install it well. Finally, as shown on the following site, I installed the old opencv with yum and installed the dependent libraries, and then removed the old opencv and it worked. https://daichan.club/linux/78330 However, there was a library that was not installed to run opnecv this time, so I installed it separately with yum.

$ sudo yum install epel-release #Add epel repository
$ sudo yum update
$ sudo yum install opencv #Put old opencv
#Dependent library installation

$ sudo yum remove opencv #Delete old opencv
$ conda install -c menpo opencv3
$ conda install -c menpo dlib
$ conda install -c meznom boost-python
$ conda install -c conda-forge tensorflow=1.1.0

#Move to the app directory

$ python web.py server
#Confirm startup and execution
#Read the error and install the required libraries
$ sudo yum install libpng12

Next, run the app with nginx + gunicorn. We will set it up referring to this article. http://qiita.com/Alice1017/items/fb28e1055c6e498d021e

setup of gunicorn and nginx

Install gunicorn. You can put it from pip, but I thought it would be better to unify it with other things, so I put it from conda.

$ conda install -c anaconda gunicorn
$ gunicorn web:app --config guniconf.py
# /tmp/gunicorn_my_app.Check if sock is generated

guniconf.py is set as follows.

guniconf.py

import multiprocessing

# Server Socket
bind = 'unix:/tmp/gunicorn_my_app.sock'
backlog = 2048

# Worker Processes
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'sync'
worker_connections = 1000
max_requests = 0
timeout = 30
keepalive = 2
debug = False
spew = False

# Logging
logfile = '/var/www/app/log/error.log'
loglevel = 'error'
logconfig = None

# Process Name
proc_name = 'gunicorn_my_app'

Next, install nginx. Since nginx in epel is old, let's register and install the nginx repository so that you can install the latest nginx yourself.

$ sudo vim /etc/yum.repos.d/nginx.repo
#Added the following contents
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/mainline/centos/7/$basearch/
gpgcheck=0
enabled=1

$ sudo yum -y --enablerepo=nginx install nginx

Edit nginx conf. Let's keep the original file just in case.

$ cd /etc/nginx/conf.d/
$ sudo mv default.conf default.conf.org
$ sudo vim default.conf

Set as follows.

default.conf

upstream my_app_server{
    server unix:/tmp/gunicorn_my_app.sock;
}

server {
  listen 80 default_server;

  error_page 404 /404.html;

  error_page 500 502 503 504 /50x.html;
  location = /50x.html {
          root /usr/share/nginx/html;
  }

  location / {
          try_files $uri @my_app;
          client_max_body_size 20M;
  }

  location @my_app {
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_redirect off;
          proxy_request_buffering off;

          proxy_pass http://my_app_server;
  }

}

client_max_body_size and proxy_request_buffering are set like this because of the app I made. We would appreciate it if you could review the settings as appropriate.

Restart nginx and see if it works.

$ sudo systemctl restart nginx
$ sudo systemctl enable nginx
# guniconf.Move to the directory where py is
$ gunicorn web:app --config guniconf.py
#Check with browser

If you can check it with a browser, stop with ctrl + c.

Introduction of supervisor

Next, I will use supervisor to make it easier to operate the gunicorn daemon. It's easiest to use yum.

$ sudo yum install supervisor
$ sudo vim /etc/supervisord.d/my_app.ini

my_app.ini

[program:my_app]
command = /home/develop/.pyenv/shims/gunicorn web:app --config /var/www/app/guniconf.py
directory = /var/www/app/
user = root 

Register the supervisor with systemd.

$ sudo vim /etc/systemd/system/supervisord.service

supervisord.service

[Unit]
Description=Supervisor process control system for UNIX
Documentation=http://supervisord.org
After=network.target

[Service]
ExecStart=/usr/bin/supervisord -n -c /etc/supervisord.conf
ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown
ExecReload=/usr/bin/supervisorctl $OPTIONS reload
KillMode=process
Restart=on-failure
RestartSec=50s

[Install]
WantedBy=multi-user.target

$ sudo systemctl list-unit-files --type=service
#Check if supervisord is registered

$ sudo systemctl start supervisord
$ sudo systemctl status supervisord
$ sudo systemctl stop supervisord
#Check if the above three are working properly

$ sudo systemctl enable supervisord.service
#Set to start automatically

#If it is set correctly, it is already working, but just in case, execute it and check
$ sudo supervisorctl reread
$ sudo supervisorctl update
$ sudo supervisorctl start my_app

You should now be able to access it from your browser.