[PYTHON] Publish your Django app on Amazon Linux + Apache + mod_wsgi

About this article

In this article, I will introduce the flow of actually deploying a web application created with Django to a production environment (= publishing it on a web server).

Environment / version to use

• OS : Amazon Linux AMI release 2018.03 --Web server: Apache / 2.4.41 (Amazon)
• Python 3.6.8
• Django 2.2.7
• mod_wsgi 4.6.8

Prerequisites

--The Django app is supposed to work in your local environment. Specifically, by using the python manage.py runserver command, it is assumed that you can already run it on localhost: 8000 without any problems. --It is assumed that you have already created an EC2 instance on AWS that allows ssh connection. --Assuming that Apache is installed in the EC2 server and the test page is displayed when accessing directly under the document root with a browser. --It is assumed that you have a Github account and that git is installed on your EC2 instance.

Prerequisite knowledge required for readers

--Can handle basic command lines. --Being able to perform basic git operations. --Have a basic knowledge of Django.

Django app deployment process

From here, we will explain the procedure for actually putting the created web application on the production environment and publishing it.

0. Directory / file structure in the server

This time, we aim to put the directory files as follows. (Note: Some irrelevant parts are omitted this time)

/
├─var
│  └─www
│     └─html  #apache document root
├─etc
│  └─httpd
│     ├─conf
│　　　│  └─httpd.conf  #apache config file
│     └─conf.d
│        └─django.conf #Additional config file to run Django
└─home
   └─ec2-user  #Here is the first directory to ssh to the instance
      └─Django
         ├─env  #The virtual environment you use to run your Django app
         └─project  #Django project
            │ manage.py
            ├─static
            ├─project
            │  │  settings.py
            │  │  local_settings.py
            │  └─ wsgi.py
            └─app
               ├─templates
               └─static

1. Upload your local Django app to Github

Creating a local / remote repository for git

This time, I'll bring the Django app inside the server via a remote repository on Github. First, create a remote repository on the Github website. After that, use the following command directly under the Django project to create a local repository linked to the created remote repository.

$ git init
$ git remote add origin [Remote repository address]

Creating a configuration file (local_settings.py) to exclude from git management

Here's something you need to do before pushing the contents of your local repository to a remote location.

The Django app has settings that need to be changed between local and remote environments. For example, in project / settings.py, there is a variable called DEBUG.

(local)project/settings.py

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

If it is True, it will display the detailed error part on the browser in case of http error. However, if this is turned on in the production environment, detailed information on the server will be leaked to a third party, so it is not good. Therefore, it is recommended to keep it False in the production environment.

Also, there is a variable called SECRET_KEY in project / settings.py.

(local)project/settings.py

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'pej^q2ql$$#bzg#crh6k_9(p=%p)&6x(kwh@nos&=!$ej&60fh' #example

This specifies the private key used for encryption and hashing, so it should not be exposed outside the project. In other words, don't push the config file with this value to a remote repository on Github.

Therefore,

--Settings that change the value between the local environment and the production environment --Settings that include confidential information (API key, DB password, etc.) that you do not want to disclose

Put the information in a file different from the normal project / settings.py and exclude it from git management.

In the project file, create a new file called local_settings.py and enter the settings you do not want to manage with git.

(local)project/local_settings.py

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'pej^q2ql$$#bzg#crh6k_9(p=%p)&6x(kwh@nos&=!$ej&60fh' #example

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

After that, delete the information described in local_settings.py from the project / settings.py file. Then, add the following line at the top to read the information in local_settings.py.

from .local_settings import *

The config file is now perfect. Then, let's make the created local_settings.py git managed.

(local)project/settings.py

$ vi .gitignore
 #Write the following
 local_settings.py

Push to remote repository

Now that you're ready to push, let's upload the source code to a remote repository.

$ git push origin master

2. Build a Python environment within an EC2 instance

Installation of 3 series Python

Python that is included in Amazon Linux by default is 2 series, so you need to install 3 series manually. This time, install Python 3.6 and the corresponding develop tool with the yum command.

$ sudo yum install python36 python36-devel

By doing this, 3 series Python will be included in the server. The caveat here is that if you just hit the command python, the default 2 system will be used. The command to use the 3 series just installed is the python3 command.

$ python -V
Python 2.7.16
$ python3 -V
Python 3.6.8

Creating a virtual environment

This time, I'll put the Django app under the / home / ec2-user / Django / folder. Therefore, create a 3rd Python virtual environment named env in this Django folder.

$ cd /home/ec2-user/Django/
$ python3 -m venv env

You have now created a virtual environment. Go inside this environment and use pip to install Django and mod_wsgi.

$ source env/bin/activate
$ pip install django mod_wsgi

If you're using other framework libraries within your Django app (for example, Django Rest Fremework), don't forget to install them as well. Below, it is assumed that you will be working in this virtual environment.

3. Place the Django app on the server

Bring your Django app from the Github remote repository you prepared in 1. Clone the remote repository under the / home / ec2-user / Django / folder.

$ git clone [Remote repository address]

You have successfully brought the app to the server. However, since there is a setting saved in local_settings.py earlier, I will manually create local_settings.py in the server.

$ vi /project/project/local_settings.py
#Write the following
SECRET_KEY = 'pej^q2ql$$#bzg#crh6k_9(p=%p)&6x(kwh@nos&=!$ej&60fh' #Make it the same value as local
DEBUG = False #Since it is a production environment, set it to False.

4. Upgrade of sqlite3

Now let's check with python manage.py runserver to see if Django works properly in the environment inside the server. At this time, if you can start without problems, skip 4 and proceed to 5. At this point, you may get a sqlite3 version error.

$ python manage.py runserver
(abridgement)
django.core.exceptions.ImproperlyConfigured: SQLite 3.8.3 or later is required (found 3.7.17).

In this case, you need to install sqlite3 with 3.8.3 or later. There was information that it wouldn't work with yum, so I decided to go steadily with wget. (Source: [django] SQLite version error)

#Get new version of sqlite3 source
$ wget https://www.sqlite.org/2019/sqlite-autoconf-3300100.tar.gz
$ tar xzf ./sqlite-autoconf-3300100.tar.gz 
$ rm -f ./sqlite-autoconf-3300100.tar.gz

#Build and install
$ cd ./sqlite-autoconf-3300100/
$ ./configure --prefix=/usr/local
$ make
$ sudo make install

Reference: What to do if SQLite3 error occurs when starting the development server in Django 2.2

Check if the installation is successful.

#Confirmation of installation destination
$ sudo find /usr/ -name sqlite3
/usr/lib64/python2.6/sqlite3
/usr/lib64/python3.6/sqlite3
/usr/lib64/python2.7/sqlite3
/usr/bin/sqlite3
/usr/local/bin/sqlite3

#Version confirmation
$ /usr/bin/sqlite3 --version
3.30.0 2019-10-10 20:19:45 18db032d058f1436ce3dea84081f4ee5a0f2259ad97301d43c426bc7f3df1b0b

I have successfully obtained sqlite3 with 3.8.3 or later. Rename the old sqlite → create a symbolic link for the new sqlite so that the sqlite3 command uses the new version.

$ sudo mv /usr/bin/sqlite3 /usr/bin/sqlite3_old
$ sudo ln -s /usr/local/bin/sqlite3 /usr/bin/sqlite3

Also, pass the path to add the new installed version of sqlite3 to the shared library.

$ vi /etc/ld.so.conf.d/sqlite3-x86_64.conf
#Write the following
/usr/local/lib

One way to pass the path to the shared library is to update the environment variable LD_LIBRARY_PATH, but even if you define the LD_LIBRARY_PATH setting in a bad place, it will not be reflected. So I quietly defined the path in ld.so.conf.d. Reference: It may not be reflected even if LD_LIBRARY_PATH is set

(Article referenced above wrote the LD_LIBRARY_PATH setting in ~ / .bashrc, but this didn't work)

5. Apache and mod_wsgi linkage settings

Once you've verified that Django can be launched successfully on your server, it's time to get ready to publish it on Apache.

First, use the find command to check the path where the mod_wsgi you entered earlier is located.

$ find -name 'mod_*.so'
./env/lib/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so

This mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so is the body of mod_wsgi. Make a note of the path you just looked up for later use.

Next, add the settings to Apache. Create a new configuration file that describes mod_wsgi with the following command.

$ sudo vi /etc/httpd/conf.d/django.conf

This will launch vi and allow you to edit the file. Here, enter the following settings.

/etc/httpd/conf.d/django.conf

LoadModule wsgi_module /home/ec2-user/Django/env/lib64/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so

WSGIScriptAlias / /home/ec2-user/Django/project/project/wsgi.py
WSGIPythonPath /home/ec2-user/Django/env/lib/python3.6/site-packages
WSGIPythonHome /home/ec2-user/Django/env

<Directory /home/ec2-user/Django/project/project>
<Files wsgi.py>
Require all granted
</Files>
</Directory>

The meaning of the setting is as follows.

--LoadModule: The location of the mod_wsgi body file. --WSGIScriptAlias: Setting to transfer to the second wsgi script when the first URL is accessed. Here, the request that came to the root of Apache is skipped to wsgi.py in the Django project. --WSGIPythonPath: The path to pass to Python during execution. --WSGIPythonHome: Python home directory for Apache to use. --Directory ~ and below: Grants Apache permissions to the specified directory.

Reference: Official document How to use Django with Apache and mod_wsgi

Once you've filled it out, restart Apache.

$ sudo service httpd restart

If you can access your site from your browser and see the project you created in Django, you're good to go!

6. Publishing static files (css, js, img)

However, I think that my project currently displayed in the browser is in a state where css and js are not working / images are not displayed. This is due to the different handling of static files between the development environment and the production environment.

Mechanism of static file publishing

In general, when developing apps with Django, static files should often be stored in static folders within each application.

project
 │ manage.py
 ├─project
 │  │  settings.py
 │  │  local_settings.py
 │  └─ wsgi.py
 ├─app1
 │  ├─templates
 │  └─static #Static files used by app1
 └─app2
    ├─templates
    └─static #Static files used by app2

However, in the production environment (= DEBUG is False in the setting), all static files are collected in one place (generally directly under the project), and the Web server (Apache) uses it. Therefore, it does not recognize when static is distributed for each application like at the time of development. Reference: Official document Deploy static files Handling static files in Django

Addition of settings.py

Django has a mechanism that allows you to do that with a single command, without having to manually create a static file directory for your production environment. From here, let's use it to add settings for collecting static files directly under the project.

Make sure you have the following settings in project / settings.py:

project/settings.py

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.2/howto/static-files/
STATIC_URL = '/static/'

This setting should be present by default. The meaning of each is as follows.

--BASE_DIR: Path of the project document root (= top project directory). --STATIC_URL: URL for static file delivery.

Once this is confirmed, add the following settings to settings.py.

project/settings.py

STATIC_ROOT = os.path.join(BASE_DIR, 'static')

STATIC_ROOT is a setting where to collect static files for production environment. Here, the setting is "Create a folder called static directly under the project".

Collecting static files

If you can do this, you are ready to go. Let's use the command to collect the static files scattered for each application in the location of STATIC_ROOT. Go directly under the project (where manage.py can be executed) and execute the following command.

$ python manage.py collectstatic

By executing this command, a static file will be automatically generated directly under the project, and all the static files used by the application will be stored in it.

project
 │ manage.py
 ├─static #New directory
 ├─project
 │  │  settings.py
 │  │  local_settings.py
 │  └─ wsgi.py
 ├─app1
 │  ├─templates
 │  └─static
 └─app2
    ├─templates
    └─static

Added settings on Apache side

Next, let Apache recognize the directories of static files gathered in one place. Add the following settings to the /etc/httpd/conf.d/django.conf file.

/etc/httpd/conf.d/django.conf

Alias /static/ /home/ec2-user/Django/project/static/

<Directory /home/ec2-user/Django/project/static>
Require all granted
</Directory>

The meaning of each is as follows.

--Alias: When the first URL is accessed, it will be transferred to the second directory. Here, when (domain name) / static (STATIC_ROOT set in settings.py) is accessed, it is skipped to the static file for the production environment created earlier with the collect static command. --Directory and below: Grant Apache permissions to static files for production environments.

Apache restart

Now that you're ready, restart Apache.

$ sudo service httpd restart

If you can access your domain from your browser and see your site with static files, it's a great success!