Sample image of Python API server for EC2 (public AMI)

A memo about my own AMI when creating an API server with Python. I don't like backdoors, so I don't think others will use them, but if anyone uses them, please do so at your own risk.

I would appreciate it if you could point out any bad settings.

It contains nginx and gunicorn.

Roughly speaking, gunicorn is started up on port 8000 using Upstart, and reverse proxy is done with nginx.

It is created assuming that multiple projects will be hosted as a development server, and the folder structure for each project is as follows.

/home/www/
└── somedomain
    ├── api_server
    │   ├── virtualenv
    │   ├── pip.txt
    │   └── python
    │       └── server.py
    └── web
        └── htdocs
            └── index.html

www is the user running * nginx * or the API server. It feels like the number of somedomain folders increases for each project.

How to start the server

The public AMI ID is ʻami-24486d4a`. I have to make sure I don't accidentally insert a different AMI.

Open the EC2 console.

Click AMI on the left menu, set the image search bar to * public image *, and enter ʻami-24486d4a` in the search field to bring up the image.

Right-click to start.

Installed

at yum

Setting

The tampered files are as follows.

Web server settings (nginx)

Server body settings

/etc/nginx/nginx.conf


user  www;
worker_processes  auto;

error_log  /var/log/nginx/error.log;

pid        /var/run/nginx.pid;


events {
    worker_connections 1024;
}

http {
    include         /etc/nginx/mime.types;
    default_type    application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;

    keepalive_timeout   65;
    types_hash_max_size 2048;

    include /etc/nginx/conf.d/*.conf;

    index   index.html index.htm;

}

I forgot to remove the option that the server version is included in the response, so I added this setting in the http directive.

    server_tokens off;

Let's update the AMI when the corrections are collected.

Virtual host settings

Even if you use CloudFlare, the original IP is set so that it does not change. For more information, click here (https://support.cloudflare.com/hc/en-us/articles/200170706-How-do-I-restore-original-visitor-IP-with-Nginx-).

__pull_from_bitbucket narrows down the access source by the IP of * Bitbucket * when Deployment cooperation with Bitbucket. If you don't use Bitbucket, turn it off.

nginx:/etc/nginx/conf.d/somedomain.conf


upstream somedomain_api_server {
    server 127.0.0.1:8000 fail_timeout=0;
}

server {
    listen       80;
    server_name  somedomain;
    root         /home/www/somedomain/web/htdocs;

    #charset koi8-r;

    # for CloudFlare
    set_real_ip_from 199.27.128.0/21;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 188.114.96.0/20; 
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 104.16.0.0/12;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 2400:cb00::/32;
    set_real_ip_from 2606:4700::/32;
    set_real_ip_from 2803:f800::/32;
    set_real_ip_from 2405:b500::/32;
    set_real_ip_from 2405:8100::/32;
    real_ip_header CF-Connecting-IP;

    access_log  /var/log/nginx/somedomain.access.log  main;

    location /__pull_from_bitbucket {
        satisfy any;
        allow 131.103.20.160/27;
        allow 165.254.145.0/26;
        allow 104.192.143.0/24;
        allow 220.156.98.58/32;
        deny all;

        try_files $uri $uri/ @proxy_to_app;
    }

    location / {
        try_files $uri $uri/ @proxy_to_app;

        add_header Cache-Tag main;
    }

    location @proxy_to_app {
        satisfy any;

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        #proxy_buffering off;
        proxy_pass   http://somedomain_api_server;
    } 
}

API server settings (gunicorn on Upstart)

GITHUB_HOOK_SECRET is an environment variable used when Deploy with GitHub. When you don't use GitHub, turn it off.

/etc/init/somedomain-api-server.conf


description "Somedomain API Server"

start on runlevel [2345]
stop on runlevel [016]

# GitHub Hook Secret
env GITHUB_HOOK_SECRET=""

respawn
exec /home/www/somedomain/api_server/virtualenv/bin/gunicorn -b 127.0.0.1:8000 -w 4 -u www --chdir /home/www/somedomain/api_server/python --log-file /home/www/somedomain/api_server/logs/error_log --pythonpath /home/www/somedomain/api_server/python server:application

Sample app

The folder for static content is web, and the folder for Python is ʻapi_server`.

Start and stop the server

start / stop nginx

#Start-up
sudo /etc/init.d/nginx start

#Stop
sudo /etc/init.d/nginx stop

#Reboot
sudo /etc/init.d/nginx restart

start / stop gunicorn

#Start-up
sudo start somedomain-api-server

#Stop
sudo stop somedomain-api-server

#Reboot
sudo restart somedomain-api-server

How to add a virtual host

  1. Become a www user, create a folder for your project, and put Python or web files on it.
  2. Set up the API server. ([Reference](http://qiita.com/polikeiji/items/51052ab6df169287a922#api%E3%82%B5%E3%83%BC%E3%83%90%E3%83%BC%E3%81%AE % E8% A8% AD% E5% AE% 9Aupstart% E3% 81% A7gunicorn))
  3. Set up the web server. ([Reference](http://qiita.com/polikeiji/items/51052ab6df169287a922#web%E3%82%B5%E3%83%BC%E3%83%90%E3%83%BC%E3%81%AE] % E8% A8% AD% E5% AE% 9Anginx))
  4. If necessary, set up GitHub and Bitbucket deployment linkage.

Recommended Posts

Sample image of Python API server for EC2 (public AMI)
Google Cloud Vision API sample for python
Image processing? The story of starting Python for
Touch the sample v20-python-samples of the OANDA v20 REST API wrapper library for Python
List of sample program distribution sites for python books
Build API server for checking the operation of front implementation with python3 and Flask
Sample usage of Python pickle
[Python] Sample code for Python grammar
Simple HTTP Server for python
Sample to use after OAuth authentication of BOX API with Python
I made a Python wrapper library for docomo image recognition API.
Image capture of firefox using python
My favorite boto3 (Python) API sample
Personal notes for python image processing
Recommended container image for Python applications
Behavior of python3 by Sakura's server
Introductory table of contents for python3
Record of Python introduction for newcomers
Memo of pixel position operation for image data in Python (numpy, cv2)
Run a batch of Python 2.7 with nohup on Amazon Linux AMI on EC2
Basics of binarized image processing with Python
Python: Basics of image recognition using CNN
PHP / Python / Ruby sample hitting Path API
Python: Application of image recognition using CNN
Summary of various for statements in Python
[Python] Calculation of image similarity (Dice coefficient)
Grayscale by matrix-Reinventor of Python image processing-
Pandas of the beginner, by the beginner, for the beginner [Python]
Sample for handling eml files in Python
Drawing with Matrix-Reinventor of Python Image Processing-
Summary of useful techniques for Python Scrapy
Analysis of X-ray microtomography image by Python
Implemented Python wrapper for Qiita API v2
Matrix Convolution Filtering-Reinventor of Python Image Processing-
Sample to convert image to Wavelet with Python
Install the Python API of the autonomous driving simulator LGSVL and execute the sample program