[GO] Run a Python web application with Docker

Last time: Operating Emby with Docker This is the trajectory of my trial and error that I want to be able to use Docker soon.

Python web application with Docker

Last time, we migrated the media streaming service Emby to containers.

This time, I created a Docker image that can deploy a Python web application. Migrate WebSocket-based and Django applications to Docker. Then move the static content to the container and close the VM server.

Static content

I moved what was on the VM server to the Docker host and bound it to the distribution directory when the container started.

docker-compose.yml

    deploy:
        image: nginx
        restart: always
        ports:
            - 80:80
            - 443:443
        volumes:
            - /home/fclef/docker/server/deploy/nginx/conf:/etc/nginx/conf.d
+           - /home/fclef/docker/server/deploy/content/html:/usr/share/nginx/html

Docker operation of Python application

Since the application using Web Socket is also made of Python (bottle), we aim to create a container that can easily deploy Python applications.

Container design

I often make web applications in Python. So, rather than consolidating the entire application into an image Imagine a foundation that can run any Python application The concrete application itself passes git information at the time of container startup, and clones and expands it in the container.

Dockerfile Whenever I write something in Python, I use Pipenv. Also, in the case of a Web application, it is made into a socket and delivered by nginx, so Install Pipenv and nginx at the time of the image.

Dockerfile

FROM python:3.8-slim-buster

#Installation of dependent packages
RUN apt-get -y update
RUN apt-get -y upgrade
RUN apt-get install -y git make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libs
qlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev nginx

#Install Pyenv
RUN git clone https://github.com/yyuu/pyenv.git /.pyenv
ENV PYENV_ROOT /.pyenv
ENV PATH $PYENV_ROOT/bin:$PATH
RUN pyenv init - | bash
#Installing Pipenv
RUN pip install pipenv
ENV PIPENV_VENV_IN_PROJECT 1
ENV PIPENV_YES true

RUN mkdir /app #Application deployment directory
RUN mkdir /script #Various script placement directories

COPY up.sh /script #Startup script

#I will clone the git project to app in the startup script, so move it
WORKDIR /app
#Web application delivery is a reverse proxy via the main web server, so leave only 80 ports open.
EXPOSE 80
RUN chmod +x /script/up.sh
ENTRYPOINT [ "/script/up.sh" ]　#When the container starts, the start script is up.run sh

Container start script

The git information of the application is passed as an environment variable when the container is started. For some reason, I can't clone git with ssh in my environment, so it's a prerequisite to clone with https. After cloning, parse the Pipfile in the project root directory to determine the Python version you need. Some Python libraries need to have dependent programs installed, so The script (dependencies.sh) for installing them is bound at the time of container startup and called here. Since gunicorn is used for socketing Python Web applications, gunicorn is also installed.

up.sh

#!/bin/bash

#Get git information from environment variables
gitCloneUrl=${GIT_CLONE_URL}
gitUserName=${GIT_USER_NAME}
gitPassword=${GIT_PASSWORD}
gitCloneTarget=`echo ${gitCloneUrl} | sed -r "s#^(https?://)(.*)#\1${gitUserName}:${gitPassword}@\2#g
"`

#Get the project name
projectName=`echo ${gitCloneUrl} | sed -r "s#.*/(\w+)\.git#\1#g"`
echo "■ ■ ■ PROJECT NAME <${projectName}> ■ ■ ■"

git clone ${gitCloneTarget}

#Get the python version from the Pipfile
cd ${projectName}
pythonVersion=`grep python_version Pipfile | sed -r 's/python_version = "(.+)"/\1/g'`
echo "■ ■ ■ PYTHON VERSION <${pythonVersion}> ■ ■ ■"

#If some of the Python libraries to be installed depend on other programs, install them here.
if [ -f /script/dependencies.sh ]; then
    source /script/dependencies.sh
fi

pipenv --python ${pythonVersion}
pipenv install
pipenv install gunicorn

curPath=`pwd`
export APP_ROOT=${curPath}

chmod +x /script/run.sh
service nginx start
/script/run.sh #Application launch script
while true; do sleep 1000; done

Start container

As an example, I'll show you a configuration that launches a Django application.

docker-compose.yml excerpt

django:
    image: pipenv-gunicorn
    restart: always
    environment:
        GIT_CLONE_URL: https://xxxxxxxx/user/project.git
        GIT_USER_NAME: user
        GIT_PASSWORD: password
    volumes:
        - /home/fclef/docker/server/app/dependencies.sh:/script/dependencies.sh
        - /home/fclef/docker/server/app/run.sh:/script/run.sh
        - /home/fclef/docker/server/app/app.conf:/etc/nginx/conf.d/nginx_gunicorn.co
nf
    depends_on:
        - deploy
        - gitlab

dependencies.sh

apt-get -y --no-install-recommends install libpq-dev

run.sh

cd /app/project

source .venv/bin/activate

python manage.py collectstatic --noinput
python manage.py makemigrations
python manage.py migrate

deactivate

/app/project/.venv/bin/gunicorn \
          --access-logfile /var/log/socket_success.log \
          --error-logfile /var/log/socket_error.log \
          --workers 1 \
          --bind unix:/run/socket.sock \
          config.wsgi:application

app.conf

server {
    listen       80;
    listen [::]:80;
    server_name xxxx.xxx.xxx;

    root /app;

    location /static {
        alias /app/project/static;
    }

    location / {
        include /etc/nginx/proxy_params;
        proxy_pass http://unix:/run/socket.sock;
    }

    location ~ ^/apple-touch-icon(.*)\.png$ {
        root /app/project/;
        rewrite ^/apple-touch-icon(.+)\.png$ /static/img/apple-touch-icon.png break;
    }
}

The application itself is cloned from git when the container is started, Since the data is stored in PostgreSQL in another container, No matter how many times you recreate this container, you can deploy the service in the operating state normally.

Basically, most Python applications can be run with the above image and startup method. Since the procedure from git clone to deployment is simple, I was able to create an image that is easy to reuse by going out of the application. For applications with complicated environment construction, it may be easier to maintain the image of each application.

This is the series for migrating VM servers to containers.