[Python/Django] Summary of frequently used commands (4) -Part 1- <Production operation: Amazon EC2 (Amazon Linux 2)>

background

Here are some frequently used commands when developing with Python/Django.

I would like to take this opportunity to thank you for solving the problem by borrowing the wisdom of our predecessors, and I am very sorry to say that I will summarize it here as my own memo.

environment

(Production environment)

AWS EC2 (Amazon Linux 2) --Python 3.7.9 * Default on Amazon Linux 2 as of 12/10/2020
Django 3.1.3 --PostgreSQL 11.5 * Same as above --Nginx 1.12 * Same as above
Gunicorn
Putty 0.74

(Development environment)

Windwos 10 Pro
Python 3.9.0
Django 3.1.3
PostgreSQL 13.1
Nginx 1.19.5
Gunicorn
Putty 0.74

Premise

Build a WEB server, AP server, and DB server on Amazon EC2 (Elastic Computing Cloud): Amazon Linux 2.

(Diagram) 構成図_基本形.png

--Execute the following commands with the SSH connection to the EC2 instance.

When connecting to Amazon EC2 (Elastic Compute Cloud) using PuTTY, please refer to "Connecting from Windows to Linux instance using PuTTY".

1. Update the Amazon Linux 2 package

`Terminal`


$ sudo yum update -y

2. Change to Japanese specifications

The default is UTC (Coordinated Universal Time), so change it to Japan Standard Time (JST).

2-1. Time zone

`Terminal`


$ sudo taimedatectl set-timezone Asia/Tokyo

$ date

Thursday, December 10, 2020 11:23:45 JST

The "date" command gets the current date and time. If it has been changed to Japan Standard Time (JST), "JST" will be displayed at the end.

2-2. Language setting

Since the default is US specification, change it to Japanese character code (jz_JP.UTF-8).

`Terminal`


$ sudo localectl set-locale LANG=ja_JP.UTF-8

$ localectl status

  System Locale: LANG=ja_JP.UTF-8
    VC Keymap: n/a
   X11 Layout: n/a

3. Create a working user and delete ec2-user

Since the default user (ec2-user) is a well-known user name, it is too high a security risk to continue using it as it is. Therefore, a new working user will be created with an arbitrary user name, and ec2-user will be deleted.

3-1. Create a working user.

`Terminal`


$sudo useradd <arbitrary user name>

3-2. Copy the SSH directory (.ssh) in the ec2-user home directory to the home directory.

`Terminal`


$ sudo cp -arp /home/ec2-user/.ssh /home/<Arbitrary user name>

3-3. Change the owner of the SSH directory and all its contents to .

`Terminal`


$ sudo chown -R <Arbitrary user name>/home/<Arbitrary user name>/.ssh

3-4. Edit sudoer (file that manages sudo authority) to grant sudo authority to .

`Terminal`


$ sudo visudo -f /etc/sudoers.d/90-cloud-init-users

`/etc/sudoers.d/90-cloud-init-users`


# User rules for ec2-user
# ec2-user ALL=(ALL) NOPASSWD:ALL comment out
<Arbitrary user name> ALL=(ALL) NOPASSWD:ALL added

3-5. Reconnect to EC2 with SSH using .

Make an SSH connection with and check if it switches to Roor (root).

`Terminal`


$ sudo su -

root #

If the display switches to "root #", it is successful. Once confirmed, type exit to log out of Root.

`Terminal`


root # exit

Log out
$

3-6. Delete ec2-user.

Also add the "-r" option to delete the ec2-user home directory.

`Terminal`


$ sudo userdel -r ec2-user

4. Install Python and check the version.

`Terminal`


$ sudo yum install python3

$ python3 --version

Python 3.7.9

5. Install the pip management module in bulk.

5-0. Output the module that was installed with pip in the development environment to "requirements.txt" in advance.

`Terminal (development environment)`


(venv_<Project name>)$ pip freeze > requirements.txt

5-1. Transfer the output "requirements.txt" to the home directory of the working user in the production environment ( above).

5-2. Create a Python virtual environment in the home directory of .

`Terminal`


$ python3 -m venv venv_<Project name>

5-3. Enter the Python virtual environment.

`Terminal`


$ source venv_<Project name>/bin/activate

5-4. Read "requirements.txt" and install the module.

`Terminal`


$ pip install -r requirements.txt

5-5. Install the module for Amazon SES (Simple Email Service).

This module is always taken care of when creating websites and web applications that use user authentication or set up inquiry forms.

`Terminal`


$ pip install boto django-ses

5-6. Install the module for Gunicorn.

`Terminal`


$ pip install gunicorn

6. Install PostgreSQL and make the initial settings.

6-1. Install PostgreSQL.

`Terminal`


$ amazon-linux-extras list | grep postgresql

  5  postgresql9.6            available    \
  6  postgresql10             available    [ =10  =stable ]
 41  postgresql11=latest      enabled      [ =11  =stable ]

It turned out that ver.11 is the highest version that can be installed. Execute the following to install.

`Terminal`


$ sudo amazon-linux-extras install postgresql11

Check the version.

`Terminal`


$ psql --version

psql (Postgres) 11.5

6-2. Install the package to operate as a DB server.

`Terminal`


$ sudo yum install postgresql-server

6-3. Make the initial settings for PostgreSQL.

6-3-1. Database initialization

`Terminal`


$ sudo postgresql-setup initdb

The following WARNING message may be displayed. This is because PostgreSQL automatically optimized the options, and there is no problem with the initialization process itself.

`Terminal`


WARNING: using obsoleted argument syntax, try --help
WARNING: arguments transformed to: postgresql-setup --initdb --unit postgresql

6-3-2. Setting to start PostgreSQL service automatically

`Terminal`


$ sudo systemctl enable postgresql

6-3-5. Start the PostgreSQL service

`Terminal`


$ sudo systemctl start postgresql

6-3-6. Log in to the PostgreSQL environment and create a user for the database and a database respectively.

`Terminal`


$ sudo -u postgres -i psql

psql (11.5)
"help"Get help with.
postgres=#create user <arbitrary user name> with password'<Arbitrary password>';
CREATE ROLE
postgres=#create database <database name> owner <arbitrary user name>;
CREATE DATABASE

6-3-7. Log out of the PostgreSQL environment.

`Terminal`


postgres=# \q

7. Place the source code in the production environment.

7-1. Install git.

`Terminal`


$ sudo yum install git

7-2. First time (when copying the whole amount)

7-2-1. Go to the Python virtual environment.

`Terminal`


$ cd ~/venv_<Project name>

7-2-2. Execute the following. * In my environment, it is necessary to enter the BitBucket password on the way.

`Terminal`


$ git clone https://<Bitbucket account>@bitbucket.org/<Bitbucket account>/<Bitbucket remote repository name>.git

Cloning into '<Remote repository name>'...
Password for 'https://<Bitbucket account>@bitbucket.org':                 
remote: Counting objects: 123, done.
remote: Compressing objects: 100% (123/123), done.
remote: Total 123 (delta 25), reused 0 (delta 0)
Unpacking objects: 100% (123/123), done.

7-3. Second time or later (when updating only the difference)

7-3-1. Go to the Python virtual environment. (Move to the directory where manage.py exists.)

`Terminal`


$ cd ~/venv_<Project name>/<Project name>

7-3-2. Execute the following. * In my environment, it is necessary to enter the BitBucket password on the way.

`Terminal`


$ git pull

Password for 'https://<Bitbucket account>@bitbucket.org':                 
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 8 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (123/123), done.
From https://bitbucket.org/<Bitbucket account>/<Bitbucket remote repository name>
  29912a3..c5aac44 master    -> origin/master
fatal: refusing to merge unrelated histories

8. Create a log output destination directory.

`Terminal`


$ mkdir ~/venv_<Project name>/<Project name>/logs

(Relation)

[Python/Django] Summary of frequently used commands (1) <Create virtual environment, project, application> [Python/Django] Summary of frequently used commands (2) [Python/Django] Summary of frequently used commands (3) [Python/Django] Summary of frequently used commands (4) -Part 2- <Production operation: Amazon EC2 (Amazon Linux 2)>

(Editor's note)

You can use EC2 as a development environment as well as during production operation, so I would like to consider using Docker as well.