[PYTHON] From environment construction to deployment for flask + Heroku with Docker

Introduction

About a year ago, I used Django to develop a simple web app.

A story about data collection, AI development, and Web application publishing all done in Python

Get data by scraping from the entered URL → Infer with a trained machine learning model → Return the answer It is a very simple thing. Click here for the app ↓ Key discrimination AI

After using it for the first time in a long time, I couldn't get the correct result due to the specification change of the scraping site. I thought this was bad and decided to repair it.

I took a peek at the code for the first time in a while, but I'm not sure ... At the time of development, I was aiming to make it work anyway, so the code is dirty and I don't even know how to deploy it ...

So, considering that it will be repaired irregularly in the future, we built an environment with ** Docker ** to make it easier to develop and deploy. The goal is to be able to develop by cloning and docker-compose, and still be able to deploy from a container to heroku.

Also, since this app itself does not use DB and is very simple, obviously there is no need to use Django, so I will rewrite it to flask at this opportunity.

I have published the code on Github. https://github.com/hatena-hanata/KJA_app

Directory structure

Here is the final directory structure.

KJA_APP
├── Dockerfile
├── Procfile
├── app.py
├── docker-compose.yml
├── requirements.txt
└── src
    ├── modules
    │   ├── module.py
    │   └── music_class.py
    ├── static
    │   └── model
    │       ├── le.pkl
    │       └── model.pkl
    └── templates

Environment

docker-compose.yml



version: '3'

services:
  web:
    build: .
    ports:
      - "8080:8080"
    volumes:
      - .:/home/KJA_APP
    tty: true
    environment:
      TZ: Asia/Tokyo
    command: flask run --host 0.0.0.0 --port 8080

Dockerfile


FROM python:3.8.0
USER root

# install google chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update && apt-get install -y google-chrome-stable && apt-get install -yqq unzip

# install chromedriver
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/

# install heroku cli
RUN curl https://cli-assets.heroku.com/install.sh | sh

# set display port to avoid crash
ENV DISPLAY=:99

# upgrade pip
RUN apt-get update && apt-get install -y \
	&& pip install --upgrade pip

# change dir
WORKDIR /home/KJA_APP

# install module
COPY requirements.txt /home
RUN pip install -r /home/requirements.txt

# flask setting
ENV FLASK_APP '/home/KJA_APP/app.py'
ENV FLASK_DEBUG 1

This time, in order to run chrome and scrape with selenium, I installed chrome and chrome driver first. I referred to the dockerfile of here. I also have heroku cli installed for deployment to heroku.

The last two lines, flask setting, are the settings for flask. In FLASK_APP, by specifying the py file that has the main function of flask, that file will be executed by the flask run command. Setting FLASK_DEBUG to 1 puts you in debug mode and allows file updates to be reflected in real time.

Now you can develop immediately with git clone → docker-compose. Since we are running the flask run command during docker-compose, you can see that the app is running by accessinghttp: // localhost: 8080 /. image.png

Deploy to Heroku

You have already registered as a member of Heroku.

Preparation

  1. requirements.txt
    Write the required libraries to requirements.txt. You need gunicorn to deploy, so don't forget to install it with pip.
  2. Procfile
    Create directly under the project. Write as follows. The app on the left is app.py directly under the project, and the app on the right is the variable name of the Flask instance defined in app.py? is. It's hard to understand what you're saying, so please read this. https://stackoverflow.com/questions/19352923/gunicorn-causing-errors-in-heroku

Procfile


web: gunicorn app:app --log-file -

App creation

You can do it from the terminal, but it's easier to understand from the browser, so I'll do it with the browser.

  1. Log in to Heroku from your browser https://id.heroku.com/login
  2. Create an app with New-> Create new app Give it a name.
  3. Add build pack To make it clear that this app is python, add heroku / python from Settings-> Buildpacks on the page of the created app. (It seems that python may be added automatically at the time of deployment, but it was a problem because it was not recognized as a python project probably because my directory structure was bad, so it is recommended to add it manually in advance. .) Also, this time we will scrape using selenium and chrome, so chrome and chrome driver are required. This can also be handled by adding a build pack. https://qiita.com/nsuhara/items/76ae132734b7e2b352dd#chrome%E3%81%A8driver%E3%81%AE%E8%A8%AD%E5%AE%9A

Deploy

From here, we will deploy in a Docker container.

  1. Log in to heroku I think a link will come up, so click on it to authenticate.
root@a59194fe1893:/home/KJA_APP# heroku login
  1. Push to heroku's app repository. The url is https://git.heroku.com/ [app name created in 2. above] .git. (You can also check it in your browser Settings). The command below means to push the contents of the current branch to the master branch of heroku's repository.
root@a59194fe1893:/home/KJA_APP# git push https://git.heroku.com/[Above 2.App name created in].git master

If there are no errors, the deployment is complete.

Finally

docker-compose is convenient.

Recommended Posts

From environment construction to deployment for flask + Heroku with Docker
From Kafka to KSQL --Easy environment construction with docker
From Python environment construction to virtual environment construction with anaconda
Virtual environment construction with Docker + Flask (Python) + Jupyter notebook
From Ubuntu 20.04 introduction to environment construction
Data science environment construction with Docker
Re: Life in Heroku starting from scratch with Flask ~ PhantomJS to Heroku ~
Vue.js + Flask environment construction memorandum ~ with Anaconda3 ~
Launch environment with LineBot + Heroku + Docker + Python
Launch Flask application with Docker on Heroku
Re: Heroku life begin with Flask from zero - Environment and Hello world -
How to upload with Heroku, Flask, Python, Git (4)
Collecting information from Twitter with Python (Environment construction)
Analytical environment construction with Docker (jupyter notebook + PostgreSQL)
Pillow environment construction --For Docker + iPython (and OpenCV)
Problems connecting to MySQL from Docker environment (Debian)
Send msgpack with ajax to flask (werkzeug) environment
From 0 to Django development environment construction to basic operation
Notes from installing Homebrew to building an Anaconda environment for Python with pyenv
Python local development environment construction template [Flask / Django / Jupyter with Docker + VS Code]
Environment construction: GCP + Docker
[Python] OpenCV environment construction with Docker (cv2.imshow () also works)
How to upload with Heroku, Flask, Python, Git (Part 3)
Output log to console with Flask + Nginx on Docker
How to upload with Heroku, Flask, Python, Git (Part 1)
Pepper-kun remote control environment construction with Docker + IPython Notebook
Python development environment construction 2020 [From Python installation to poetry introduction]
How to upload with Heroku, Flask, Python, Git (Part 2)
Environment construction with anyenv + pyenv (migrate from pyenv only (Mac))
Example of pytest environment to fix database with Docker
Procedure to exe python file from Ubunts environment construction
Environment construction for those who want to study python easily with VS Code (for Mac)
[With image diagram] Nginx + gunicorn + Flask converted to Docker [Part 2]
Execute C ++ functions from Python with Pybind11 (for Windows & Visual Studio Code people) Environment construction
How to deploy a web app made with Flask to Heroku
OpenJTalk on Windows10 (Speak Japanese with Python from environment construction)
Touch Flask + run with Heroku
[Memo] Build a development environment for Django + Nuxt.js with Docker
ML environment construction with Miniconda
Docker + Django + React environment construction
Prepare python3 environment with Docker
Python3 environment construction (for beginners)
I created an environment for Masonite, a Python web framework similar to Laravel, with Docker!
Introduction to Python for VBA users-Calling Python from Excel with xlwings-
[With image diagram] Nginx + gunicorn + Flask converted to Docker [Part 1]
Ansible environment construction For Mac
Until API made with Flask + MySQL is converted to Docker
[AWS] Flask application deployment version that tried to build a Python environment with eb [Elastic Beanstalk]
Web application created with Python + Flask (using VScode) # 1-Virtual environment construction-
Python environment construction 2016 for those who aim to be data scientists
How to debug Dash (Flask) in Docker + VSCode + remote connection environment
From building a Python environment for inexperienced people to Hello world
How to use jupyter notebook without polluting your environment with Docker
Spigot (Paper) Introduction to how to make a plug-in for 2020 # 01 (Environment construction)
(Note) Notes on building TensorFlow + Flask + Nginx environment with Docker Compose
Create a development environment for Go + MySQL + nginx with Docker (docker-compose)
I made a development environment for Django 3.0 with Docker, Docker-compose, Poetry
Postgres environment construction with Docker I struggled a little, so note
It's time to install DB with Docker! DB installation for beginners on Docker
From ubuntu installation to running kinect with docker and ros (overview)
From PyCUDA environment construction to GPGPU programming on Mac (MacOS 10.12 Sierra)