[PYTHON] Using Flask with Nginx + gunicorn configuration [Local environment]

Summary of this article

• As Step 1 of Web application learning roadmap with Python * ** Describes the procedure for launching a Flask application with the following configuration in the local environment and related settings **. (See the figure below for the configuration) It assumes a Python installed Mac OS. (But I don't think there is a big difference on other platforms.)

Article quick reference table

――When you read a similar article, why often? What do you mean? I feel-> [What is / what is ○○? ](What does # mean) --I should move-> [Try to move](# Try to move) --I want to know about the configuration file of gunicorn and Nginx-> [Unravel the configuration file and start / end commands](#Unravel the configuration file start / end commands)

Try to move

First, let's actually operate with the above configuration. Install the required packages.

brew install nginx
pip install gunicorn

Project structure and contents of each file

PROJECT
├── config
│   ├── gunicorn_settings.py #Application server configuration file
│   └── nginx.conf           #Web server configuration file
└── flask_app.py             #Main application

--Application file
This time, we don't care about the behavior of the application, so we will prepare only two routes to return the character string.

flask_app.py

from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
    return 'Index Page!'

@app.route('/health')
def health():
    return 'Health Check OK!!'

--Web server configuration file

gunicorn_settings.py

import os

bind = '127.0.0.1:' + str(os.getenv('PORT', 9876))
proc_name = 'Infrastructure-Practice-Flask'
workers = 1

--Nginx configuration file

nginx.conf

#See the configuration file section for details
worker_processes  1;

events {
	worker_connections 512;
}

http {
    server {
    	listen  9123;
    	server_name INFRA-PRACTICE-NGINX;
		charset UTF-8;

		proxy_set_header    Host    $host;

    	location / {
        	proxy_pass http://127.0.0.1:9876;
    	}

	    location /health/ {
        	proxy_pass http://127.0.0.1:9876/health;
    	}
    }
}

Start a process

Terminal

#Start the web server
nginx -c ${PROJECT_PATH}/config/nginx.conf

#Start application server
gunicorn flask_app:app -c config/gunicorn_settings.py

Confirm communication

Terminal

curl http://localhost:9123/
# -> Index Page!OK if it returns
curl http://localhost:9123/health/
# -> Health Check OK!!OK if it returns

Unravel the configuration file and start / end commands

gunicorn

setting file

In the Official Document, it is described in the optional format. When using a configuration file, use the key = value format with the configuration item name as the key (* e.g. * loglevel ='debug'). The extension of the configuration file is arbitrary. Some of the settings are shown below.

Start / end command

【Start-up】

Terminal

gunicorn {Flask application file name}:{Flask instance variable name in the file} -c {Configuration file path}

• As mentioned in the section of the configuration file, Setting can also be specified as a startup option.

[End] Ctrl + C in Terminal where process is running

Nginx

setting file

Looking at the official docs, it's different from gunicorn's key = value format. You can see that the Directive (instruction) is described in the context format (* curly bracket: * block enclosed in {}) inside the Module.

Module

• In addition, there are mail and stream.

Directive Syntax in http context A server context within the http context and a further location context within the server context And so on, we will write in context in context format. Here is an example of setting the http block. By looking at the example, you can understand what the configuration file at the beginning means.

--Basic settings

config

http {
    server {
        listen        80;                          #Wait for connection on port 80
        server_name   MY_SERVER;                   #The server name is MY_SERVER
        charset       UTF-8;                       #Content of response header-type to UTF-Designated as 8
        error_page    404  /404_not_found.html;    #404 when status code is 404_not_found.Internally redirect to html
    }
}

--Reverse proxy

config

http {
    server {
        # /Access to index
        location /index {
            proxy_pass          http://127.0.0.1:8080;  # 127.0.0.Reverse proxy to port 1 8080
            proxy_set_header    Host    $host;          #Of the request headers passed to the proxied server, host$Reconfigure host
　　　　　　  proxy_read_timeout  60s;                    #Time out if there is no response from the proxied server for 60 seconds
        }
    }
}

--Access control / IP filtering

config

config
http {
    server {
        location / {
            allow 192.168.0.1;    # 192.168.0.Allow access from 1
            deny  all;            #Deny any other network access
        }
    }
}

--BASIC authentication

config

http {
    server {
        location / {
            auth_basic             "Basic Auth closed site";　　#Clarify that BASIC authentication is required
            auth_basic_user_file   conf/htpasswd;               # .htpasswd Specify the file path
        }
    }
}

--Referer check

config

http {
    server {
        #There is no Referer in the request header(none), In the header but http://Or https://Does not start with(blocked)、
        # mintak.Does not end with com(server_names)Judged as invalid when accessing from the root->$invalid_Set referer to 1
        valid_referers none blocked server_names *mintak.com
        if ($invalid_referer) {
           return 403;  #If it is determined to be invalid, return it with 403 Forbidden
        }
    }
}

Embbed Variable Something like an environment variable in an Nginx server. For more information, click here (https://nginx.org/en/docs/http/ngx_http_core_module.html#variables). Some are shown below.

Start / end command

【Start-up】

Terminal

nginx -c {Configuration file path}

[End]

Terminal

nginx -s stop

#If the above fails, identify the process and kill
ps ax | grep nginx
kill -9 ${PID}

[Startup confirmation]

Terminal

ps aux | grep nginx    #Here nginx:If you see master process, nginx is running.

#Command that can confirm PORT
lsof -i -P | grep nginx
#=> nginx     41845 mintak    6u  IPv4 0x595e7f947462ada5      0t0  TCP *:9123 (LISTEN)

What is ○○?

You can achieve the purpose by preparing the files up to this point and hitting the command. However, there are many difficult words and concepts for beginners like myself. From here, I will describe my own interpretation of such keywords.

Why is it such a configuration

According to an expert, the Web server and application server were not separated in the past. This configuration became a problem when we faced the fact that the load on the server increased due to the increase in the number of accesses to services that are open to the public due to the spread of the Internet, and the overall performance deteriorated. did. Therefore, aiming for stable operation by distributing the load, the flow is to divide the server according to the role as described later. This idea is commonplace in modern times, and it seems that Web services are generally made up of a three-layer structure of Web server <-> application server <-> DB server. I think that the basic idea of ** separating processes (servers) for each role ** is similar to the design concept for coding (one clear role for each class / function).

But it works with python flask_app.run

This is because the framework called Flask is a framework that can be started while concurrently serving as a Web server / application server. It is useful when it is difficult to set up the Web server and the application server separately, such as ** for simple verification and operation check during development **. When starting with this command, the log "WARNING: This is a development server. Do not uses it in a production development.`" appears [^ 1], but it is for development only, and with this configuration, mass access It means that there is a big risk of a flat tire in the process, so ** stop in production **.

• If you add ʻapp.run (host = '0.0.0.0, port = 9876)` to flask_app.py, it will start up even with the above command.

The main role of the web server and application server

I will summarize each in one word. See the Reference link page for details.

--Web server: Supports requests that do not change regardless of who accesses them (static processing). The TOP page is easy to hit. --Application server: Supports applications (dynamic processing) whose response changes according to access. Member My Page, etc. (Display contents differ depending on the member).

Why is a reverse proxy reverse?

If there is a reverse, there is a forward **. What we usually call a proxy is a forward proxy.

What is the difference between a forward proxy and a reverse proxy?

They are still proxies to each other. Processing between the client-> Internet is a forward proxy (server), and processing between the Internet-> target Web server is a reverse proxy (server). [^ 2]

You can connect it directly to the target server! You might think, but you can benefit from biting a proxy server (see below).

Use of forward proxy

Use of reverse proxy

What is WSGI?

Abbreviation for Web Server Gateway Interface. ~~ Commonly known as Uesugi-kun. ~~ Django and Flask are frameworks that support WSGI, and gunicorn and uWEGI are servers that support WSGI, so I think there is no problem with recognition. The background of its appearance is as follows. [^ 3]

・ Framework A-chan> I'm cool, so I'm crazy except α-kun, a cool web server ・ Framework B-chan> I like the solid β-kun ♡

In the past, the web servers that can be used for each Python web framework were limited (and vice versa). Actually, I wanted B-chan to use α-kun, but here the marriage-like relationship with A-chan is a drag. It was ~~ Uesugi-kun and ~~ WEGI who appeared to solve this problem. WEGI says, "Let's freely choose each other for the server and framework ~~ love ~~ Let's connect", how to interact in the relationship of application framework <-> Web server I set the rule. Thanks to WSGI and WSGI-compatible servers and frameworks, we are now free to choose the combination of server and framework. I'm happy.

References -Gunicorn Official Document -Nginx document (original) -Nginx document (Japanese translation) -Summary of Nginx settings (Qiita) -Difference between Web server and AP server (Think IT)

[^ 1]: If you haven't set FLASK_ENV = development. [^ 2]: This time, Nginx is both a proxy server and a web server. [^ 3]: I think this is the only way to explain this stupid thing. .. ..