[PYTHON] From environment construction to deployment for flask + Heroku with Docker

Introduction

About a year ago, I used Django to develop a simple web app.

A story about data collection, AI development, and Web application publishing all done in Python

Get data by scraping from the entered URL → Infer with a trained machine learning model → Return the answer It is a very simple thing. Click here for the app ↓ Key discrimination AI

After using it for the first time in a long time, I couldn't get the correct result due to the specification change of the scraping site. I thought this was bad and decided to repair it.

I took a peek at the code for the first time in a while, but I'm not sure ... At the time of development, I was aiming to make it work anyway, so the code is dirty and I don't even know how to deploy it ...

So, considering that it will be repaired irregularly in the future, we built an environment with ** Docker ** to make it easier to develop and deploy. The goal is to be able to develop by cloning and docker-compose, and still be able to deploy from a container to heroku.

Also, since this app itself does not use DB and is very simple, obviously there is no need to use Django, so I will rewrite it to flask at this opportunity.

I have published the code on Github. https://github.com/hatena-hanata/KJA_app

Directory structure

Here is the final directory structure.

KJA_APP
├── Dockerfile
├── Procfile
├── app.py
├── docker-compose.yml
├── requirements.txt
└── src
    ├── modules
    │   ├── module.py
    │   └── music_class.py
    ├── static
    │   └── model
    │       ├── le.pkl
    │       └── model.pkl
    └── templates

Environment

docker-compose.yml

version: '3'

services:
  web:
    build: .
    ports:
      - "8080:8080"
    volumes:
      - .:/home/KJA_APP
    tty: true
    environment:
      TZ: Asia/Tokyo
    command: flask run --host 0.0.0.0 --port 8080

Dockerfile

FROM python:3.8.0
USER root

# install google chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update && apt-get install -y google-chrome-stable && apt-get install -yqq unzip

# install chromedriver
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/

# install heroku cli
RUN curl https://cli-assets.heroku.com/install.sh | sh

# set display port to avoid crash
ENV DISPLAY=:99

# upgrade pip
RUN apt-get update && apt-get install -y \
	&& pip install --upgrade pip

# change dir
WORKDIR /home/KJA_APP

# install module
COPY requirements.txt /home
RUN pip install -r /home/requirements.txt

# flask setting
ENV FLASK_APP '/home/KJA_APP/app.py'
ENV FLASK_DEBUG 1

This time, in order to run chrome and scrape with selenium, I installed chrome and chrome driver first. I referred to the dockerfile of here. I also have heroku cli installed for deployment to heroku.

The last two lines, flask setting, are the settings for flask. In FLASK_APP, by specifying the py file that has the main function of flask, that file will be executed by the flask run command. Setting FLASK_DEBUG to 1 puts you in debug mode and allows file updates to be reflected in real time.

Now you can develop immediately with git clone → docker-compose. Since we are running the flask run command during docker-compose, you can see that the app is running by accessinghttp: // localhost: 8080 /.

Deploy to Heroku

You have already registered as a member of Heroku.

Preparation

1. requirements.txt
   Write the required libraries to requirements.txt. You need gunicorn to deploy, so don't forget to install it with pip.
2. Procfile
   Create directly under the project. Write as follows. The app on the left is app.py directly under the project, and the app on the right is the variable name of the Flask instance defined in app.py? is. It's hard to understand what you're saying, so please read this. https://stackoverflow.com/questions/19352923/gunicorn-causing-errors-in-heroku

Procfile

web: gunicorn app:app --log-file -

App creation

You can do it from the terminal, but it's easier to understand from the browser, so I'll do it with the browser.

1. Log in to Heroku from your browser https://id.heroku.com/login
2. Create an app with New-> Create new app Give it a name.
3. Add build pack To make it clear that this app is python, add heroku / python from Settings-> Buildpacks on the page of the created app. (It seems that python may be added automatically at the time of deployment, but it was a problem because it was not recognized as a python project probably because my directory structure was bad, so it is recommended to add it manually in advance. .) Also, this time we will scrape using selenium and chrome, so chrome and chrome driver are required. This can also be handled by adding a build pack. https://qiita.com/nsuhara/items/76ae132734b7e2b352dd#chrome%E3%81%A8driver%E3%81%AE%E8%A8%AD%E5%AE%9A

Deploy

From here, we will deploy in a Docker container.

1. Log in to heroku I think a link will come up, so click on it to authenticate.

root@a59194fe1893:/home/KJA_APP# heroku login

2. Push to heroku's app repository. The url is https://git.heroku.com/ [app name created in 2. above] .git. (You can also check it in your browser Settings). The command below means to push the contents of the current branch to the master branch of heroku's repository.

root@a59194fe1893:/home/KJA_APP# git push https://git.heroku.com/[Above 2.App name created in].git master

If there are no errors, the deployment is complete.

Finally

docker-compose is convenient.