[PYTHON] Build NGINX + NGINX Unit + MySQL environment with Docker

Introduction of Docker

Previously, I built an environment of NGINX + NGINX Unit + Flask.

Running Python web apps with NGINX + NGINX Unit + Flask

Last time I built it with Vagrant and VirtualBox, but this time I will build it with Docker. As before, use NGINX for the web server, NGINX Unit for the AP server, and Flask for the framework. Furthermore, this time we will add MySQL as a database and build an environment for WEB <-> AP <-> DB.

The image is as shown in the figure below. (Originally the container starts in the host, but it is separated for clarity)

We will build it step by step, so if you want to see what you have done, please check it out on GitHub.

https://github.com/txkxyx/docker-web

environment

We will build in the following environment.

--Host - OS : macOS Catalina 10.15.3 - Docker : 19.03.5 --Container - Python : 3.7.3 - Flask : 1.1.1 - NGINX : 1.17.7 - NGINX Unit : 1.14.0 - MySQL : 8.0.18 - Flask SQLAclchemy : 2.4.1

The directory structure is as follows.

./web
    |- db                   //For DB
    |- nginx                //For NGINX
    |- python               //For NGINX Unit and source files
    |    |- src
    |- docker-compose.yml

Let's get started.

Dockerfilet and docker-compose settings

Here is a brief summary of the settings used in Dokerfile and docker-compose.

Dockerfile settings

See the official Dockerfile reference for more information.

https://docs.docker.com/engine/reference/builder/

docker-compose settings

See the official reference for more details.

https://docs.docker.com/compose/compose-file/

Building a DB container

First, we will build a MySQL container. The image looks like this.

Create docker-compose.yml.

web/docker-compose.yml

version: "3"

services:
    db:
        image: mysql
        command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
        ports:
            - "33306:3306"
        expose:
            - "3306"
        environment:
            MYSQL_ROOT_PASSWORD: root
            MYSQL_USER: test
            MYSQL_PASSWORD: test
        volumes:
            - ./db/init:/docker-entrypoint-initdb.d
        container_name: app_db

Create a ʻinitdirectory inside thedb` directory and create createdatabase.sql, just as you would create a database on initial container startup.

web/db/init/createdatabase.sql

CREATE DATABASE app;
USE app;

CREATE TABLE users(
    id INT PRIMARY KEY AUTO_INCREMENT,
    name VARCHAR(255),
    email VARCHAR(255)
);

INSERT INTO users(name,email) VALUES('sample','[email protected]');
INSERT INTO users(name,email) VALUES('test','[email protected]');
INSERT INTO users(name,email) VALUES('app','[email protected]');

GRANT ALL ON app.* TO test;

With the above settings, start the MySQL container with docker-compose.

$ docker-compose up
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                NAMES
bef9a864276c        mysql               "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes        33060/tcp, 0.0.0.0:33306->3306/tcp   app_db

If ʻapp_db is displayed in the result of docker ps`, the container has been started. Once inside the container, check if the database has been created.

$ docker exec -it app_db bash
root@00000000000:/# mysql -u test -p
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| app                |
| information_schema |
+--------------------+
2 rows in set (0.00 sec)
mysql> use app;
mysql> select * from users;
+----+--------+-------------------+
| id | name   | email             |
+----+--------+-------------------+
|  1 | sample | [email protected] |
|  2 | test   | [email protected]     |
|  3 | app    | [email protected]       |
+----+--------+-------------------+
3 rows in set (0.01 sec)

You can see that a database called ʻapphas been created. In addition, you can see that the table and data forcreatedatabase.sql` have been created. That's all for building MySQL.

Building an AP container

Build a container using NGINX Unit as the AP server, Python3 as the execution environment, and Flask as the framework. We will build it by referring to the Official Document of NGINX Unit. The image looks like this.

Launching NGINX Unit container

First, build the environment of Python3 and Flask from the image of NGINX Unit. This may be the smallest unit of the development environment. Add Dockerfile to the web / python directory.

web/python/Dorckerfile

FROM nginx/unit:1.14.0-python3.7

WORKDIR /usr/src/app

COPY src .

RUN apt update && apt install -y python3-pip                               \
    && pip3 install --no-cache-dir -r ./requirements.txt                            \
    && rm -rf /var/lib/apt/lists/* 

CMD ["sleep","infinity"]

From the Docker Hub nginx / unit site, use the image of NGINX Unit for Python 3.7.

Next, create requirements.txt in the web / python / src directory so that you can install the libraries in bulk with pip.

web/python/src/requirements.txt

Flask == 1.1.1
flask-sqlalchemy == 2.4.1
PyMySQL == 0.9.3

Add the NGINX Unit container settings to docker-compose.yml.

docker-compose.yml

version: "3"

services:
    db:
        image: mysql
        ports:
            - "33306:3306"
        expose:
            - "3306"
        environment:
            MYSQL_ROOT_PASSWORD: root
            MYSQL_USER: test
            MYSQL_PASSWORD: test
        volumes:
            - ./db/init:/docker-entrypoint-initdb.d
        container_name: app_db
    #↓ ↓ Addendum
    ap:
        build: ./python
        ports:
            - "8080:8080"
        environment:
            TZ: "Asia/Tokyo"
        container_name: app_ap
        depends_on:
            - db

The created'Dockerfile'exists in the web / python directory, so specify the location with build. The server port should expose 8080 to the host. Stop the running Docker container, then build with docker-compose build and then start the container.

$ docker-compose down
$ docker-compose build --no-cache
$ docker-compose up
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                NAMES
daf4ddc7c11a        web_ap              "sleep infinity"         41 seconds ago      Up 40 seconds       0.0.0.0:8080->8080/tcp               app_ap
565eb32e6a39        mysql               "docker-entrypoint.s…"   43 seconds ago      Up 41 seconds       33060/tcp, 0.0.0.0:33306->3306/tcp   app_db

You can see that the MySQL container ʻapp_db and the NGINX Unit container ʻapp_ap are running. Go into the NGINX Unit container and check if the requirements.txt library is installed.

$ docker exec -it app_ap bash
root@00000000000:/# python3 -V
Python 3.7.3
root@00000000000:/# pip3 freeze
Flask==1.1.1
Flask-SQLAlchemy==2.4.1
PyMySQL==0.9.3

In addition to the above libraries, SQLAlchemy and Jinja2 are installed. This completes the startup of the NGINX Unit container. Next, we will implement Flask.

Flask application implementation

Implement the Flask application. The files and directories to be created are as follows.

./web
    |- db
    |   |- init
    |       |- createdatabase.sql
    |- nginx  
    |- python
    |   |- src
    |   |   |- app.py ← added
    |   |   |- config.json ← added
    |   |   |- config.py ← added
    |   |   |- run.py ← added
    |   |   |- users.py ← added
    |   |   |- requirements.txt
    |   |   |-templates ← added
    |   |       |- list.html ← added
    |   |-Dockerfile ← Update
    |- docker-compose.yml

Each file is implemented as follows.

config.py

The first is config.py, which implements the configuration class such as the DB connection destination. The host destination is specified by the container name ʻapp_db` of the DB container.

web/python/src/config.py

class Config(object):
    '''
    Config Class
    '''
    # DB URL
    SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://test:test@app_db:3306/app?charset=utf8'

app.py

Next is ʻapp.py, which launches the Flask application. Use config.from_object ()to call the application's configuration class, and useSQLAlchemy ()` to initialize it so that your Flask application can use SQLAchemy.

web/python/src/app.py

from config import Config
from flask import Flask
from flask_sqlalchemy import SQLAlchemy

# Create Flask Application
application = Flask(__name__)

# Set Config Class
application.config.from_object(Config)

# Set DB
db = SQLAlchemy(application)

users.py

Next, create the Model class for the users table. Create a Users class that inherits from the db.Model class.

web/python/src/users.py

from app import db

class Users(db.Model):
    '''
    Users Table Model
    '''
    __tablename__ = 'users'
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(255))
    email = db.Column(db.String(255))

    def __init__(self,name,email):
        self.name = name
        self.email = email

run.py

Next is the Flask application launch and routing module run.py. Since the template file is used for the response, specify the template file and object with render_template ().

web/python/src/run.py

from app import application
from users import Users
from flask import render_template

@application.route('/list')
def index():
    users = Users.query.order_by(Users.id).all()
    return render_template('list.html', users=users)

if __name__ == '__main__':
    application.run(host='0.0.0.0', port='8080')

list.html

Next, create a template file, list.html. Since render_template () passes a ʻusers object, implement it using the template engine Jinja2`.

web/python/src/templates/list.html

<!DOCTYPE html>
<html lang="ja">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Flask Sample</title>
</head>
<body>
    <h1>Flask Sample</h1>
    <table border="1" style="border-collapse: collapse">
        <thead>
            <tr>
                <th >Id</th>
                <th >Name</th>
                <th >EMail</th>
            </tr>
        </thead>
        <tbody>
            {% for user in users %}
                <tr>
                    <td>{{user.id}}</td>
                    <td>{{user.name}}</td>
                    <td>{{user.email}}</td>
                </tr>
            {% endfor %}
        </tbody>
    </table>
</body>
</html>

Dockerfile

Update Dockerfile.

web/python/Dorckerfile

FROM nginx/unit:1.14.0-python3.7

WORKDIR /usr/src/app

COPY src .

RUN apt update && apt install -y python3-pip                               \
    && pip3 install --no-cache-dir -r ./requirements.txt                            \
    && rm -rf /var/lib/apt/lists/*
#↓ ↓ Delete

config.json

Finally, add the NGINX Unit configuration file config.json.

web/python/src/config.json

{
  "listeners": {
    "*:8080": {
      "pass": "applications/app"
    }
  },

  "applications": {
    "app": {
      "type": "python",
      "processes": 2,
      "path": "/usr/src/app/",
      "module": "run"
    }
  }
}

That's all for the implementation.

Let's build and then start the container.

$ docker-compose down
$ docker-compose build --no-cache
$ docker-compose up
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                NAMES
daf4ddc7c11a        web_ap              "sleep infinity"         41 seconds ago      Up 40 seconds       0.0.0.0:8080->8080/tcp               app_ap
565eb32e6a39        mysql               "docker-entrypoint.s…"   43 seconds ago      Up 41 seconds       33060/tcp, 0.0.0.0:33306->3306/tcp   app_db

After starting the ʻapp_ap` container, access the container and set the NGINX Unit configuration file.

$ docker exec -it app_ap bash
root@00000000000:/# curl -X PUT --data-binary @config.json --unix-socket /var/run/control.unit.sock http://localhost/config
{
	"success": "Reconfiguration done."
}

In your browser, go to http: // localhost: 8080 / list and you will see the screen.

This completes the construction of the AP container.

Building a web container

Finally, we will build the NGINX container for the WEB server. This will configure NGINX <-> NGINX Unit <-> Flask <-> MySQL.

The files to be added or updated are as follows.

./web
    |- db
    |   |- init
    |       |- createdatabase.sql
    |- nginx
    |   |-Dockerfile ← added
    |   |- index.html ← added
    |   |- nginx.conf ← added
    |- python
    |   |- src
    |   |   |- __init__.py
    |   |   |- app.py
    |   |   |- config.json
    |   |   |- config.py
    |   |   |- run.py
    |   |   |- users.py
    |   |   |- requirements.txt
    |   |   |- templates
    |   |       |- index.html
    |   |- Dockerfile
    |- docker-compose.yml ← update

First, create the top page ʻindex.html`.

web/nginx/index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Index</title>
</head>
<body>
    <h1>Index</h1>
    <a href="/list">List</a>
</body>
</html>

Next, create a Dockerfile for NGINX.

FROM nginx

WORKDIR /var/www/html

COPY ./index.html ./

CMD ["nginx", "-g", "daemon off;","-c","/etc/nginx/nginx.conf"]

Next, create a NGINX configuration file. From the config file introduced in the previous article, change the host of the AP server for Docker.

nginx.conf

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;
    server_tokens off;

    keepalive_timeout  65;

    #gzip  on;

    upstream unit-python {
        server app_ap:8080; # container_Specified by name
    }
    server {
        listen 80;
        server_name localhost;

        #Display top page
        location  / {
            root /var/www/html;
        }

        # /list routes to AP container
        location  /list {
            proxy_pass http://unit-python;
            proxy_set_header Host $host;
        }
    }
}

Finally update docker-compose.

docker-compose.yml

version: "3"

services:
    db:
        image: mysql
        ports:
            - "33306:3306"
        expose:
            - "3306"
        environment:
            MYSQL_ROOT_PASSWORD: root
            MYSQL_USER: test
            MYSQL_PASSWORD: test
        volumes:
            - ./db/init:/docker-entrypoint-initdb.d
        container_name: app_db

    ap:
        build:
            context: ./python
            args:
                project_directory: "/src/"
        #↓ ↓ update
        expose:
            - "8080"
        volumes:
            - "./python/src:/projects"
        environment:
            TZ: "Asia/Tokyo"
        container_name: app_ap
        depends_on:
            - db
    #↓ ↓ added
    web:
        build: ./nginx
        volumes:
            - ./nginx/nginx.conf:/etc/nginx/nginx.conf
        ports:
            - "80:80"
        environment:
            TZ: "Asia/Tokyo"
        container_name: "app_web"
        depends_on:
            - ap

Let's build and then start the container.

$ docker-compose down
$ docker-compose build --no-cache
$ docker-compose up
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                NAMES
5b0f06b89db4        web_web             "nginx -g 'daemon of…"   2 minutes ago       Up 23 seconds       0.0.0.0:80->80/tcp                   app_web
625f3c025a82        web_ap              "/usr/local/bin/dock…"   2 minutes ago       Up 2 minutes        8080/tcp                             app_ap
fe5bf54411a2        mysql               "docker-entrypoint.s…"   2 minutes ago       Up 2 minutes        33060/tcp, 0.0.0.0:33306->3306/tcp   app_db

After starting the app_ap container, access the container and set the NGINX Unit configuration file.

$ docker exec -it app_ap bash
root@00000000000:/# curl -X PUT --data-binary @config.json --unix-socket /var/run/control.unit.sock http://localhost/config
{
	"success": "Reconfiguration done."
}

If you access http: // localhost: 80 with a browser, the top page ʻindex.html` will be displayed. (Not accessible on port 8080)

If you press the linked List, the Flask application list.html will be displayed.

This is the end of NGINX construction.

Summary

I was able to build an environment of NGINX + NGINX Unit + MySQL with Docker. All you have to do is build the application.