[PYTHON] [AWS] I tried using EC2, RDS, Django. Environment construction from 1

Introduction

This was my first time touching the back end, which was quite annoying. The next time I created it, I thought that I could create it immediately after seeing this article, so I created it for myself or for those who are similarly confused **.

Since I am a beginner, there may be some mistakes or strange ideas. In that case, I would appreciate it if you could let me know in the comment section.

$             <-Commands on my PC terminal
[ec2-user] $  <-Commands while logged in to EC2
MySQL >       <-Commands logged in to MySQL
#             <-My comment
>>>           <-Execution result(Output value)

Prerequisites

--AWS account created

[1] AWS Preferences

[1.1] Create VPC

VPC is a service that provides a private cloud environment dedicated to users. For example, many AWS use VPCs when you want to communicate EC2 internally, communicate with RDS, and connect internal and external networks.

** Creation procedure **

-[1] Creating a VPC -[2] Name tag "VPCtest" -[3] IPv4 CIDR block "10.0.0.0/16" -[4] Set the tag key to "Name" and the value to "VPCtest"

This completes the VPC creation.

[1.1.2] Subnet creation

A subnet is a group of networks that first divides a CIDR block created by a VPC. You can create as many subnets as you like, as long as you don't exceed the VPC limit. There are mainly ** public subnets ** and ** private subnets **.

I think it's hard to get an image at first, so ... I created a ** VPC ** called ** Home ** in a vast ** Earth ** Internet space . The image is that a room with a different role ( subnet **) such as ** bath ** or ** bedroom ** was created there. I'm sorry if I think wrong (； ´Д ｀)

Create public and private subnets. ** EC2 ** for public subnets Put ** RDS ** in the private subnet. Create two ** private subnets ** because RDS needs to ** have different Availability Zones **.

** Creation procedure **

--Create subnet -[1] Create public test -[1.1] Put the "VPCtest" created earlier in the VPC -[1.2] Select "1a" for the Availability Zone for the time being -[1.3] IPv4 CIDR block is "10.0.1.0/24" -[2] Create private test -Same as [2.1] [1.1] [1.2] -[2.2] IPv4 CIDR block is "10.0.2.0/24" -[3] Create private test2 -Same as [3.1] [1.1] -[3.2] Select "1c" for the availability zone (OK if the availability zone is different from [2.1]) -[3.3] IPv4 CIDR block is "10.0.200.0/24"

This completes the subnet settings.

[1.1.3] Route table creation

The route table defines the rules for where the instances in the subnet go to communication. In other words, the route table is a table that looks at the destination (IP address) of the packet and describes where to send the communication. Since it carries packets by looking at this table, it cannot communicate with destinations that are not in the table because it does not send packets. The point is that it defines where communication can be performed for each subnet.

** Creation procedure **

-[1] Creating a route table -[2] Name tag "root test" -[3] Put the "VPCtest" created earlier in the VPC

Next is the creation of the Internet gateway.

[1.1.4] Creating an internet gateway

A gateway for connecting to the Internet from within a VPC. By using this, the system in the VPC can use the global IP.

For example, it is the role of ** entrance security ** (key) that puts ** people ** (Internet) into ** home ** (VPC area).

** Creation procedure **

-[1] Internet gateway creation -[2] Set the name tag to "igw-test" -[3] Set the tag key to "Name" and the value to "igw-test" -[4] Attach to VPC -[5] Put the "VPCtest" created earlier in the VPC

This completes the Internet gateway settings.

[1.1.5] Added external route to route table

** Creation procedure **

-[1] Edit route -[2] Press Add Route -[3] Set the destination to "0.0.0.0/0" -[4] Select the Internet gateway created this time from Internet Gateway as the target. -[5] Save route

This completes the route table settings.

[1.1.6] Associate a route table with a subnet

Edit the route table association.

** Creation procedure **

-[1] Edit public test route table association -[2] Change the route table ID to the "route test" created this time. -[3] Save

[1.2] Create an EC2 instance

** Creation procedure **

-[1] Launch an instance -[2] Select Amazon Linux2 AMI (HVM), SSD Volume Type -[3] After selecting the instance type of the free tier, go to the next step (detailed settings) -[4] After "VPCtest" for the network, "Public test" for the subnet, and "Enable" for the auto-assigned IP, go to the next step (additional storage) -[5] Go to the next step (adding a tag) -[6] Add another tag, set the tag key to "Name" and the value to "web server test", and then go to the next step (security group). -[7] Create a new security group and name it "webtest-sg" -[7] Download the key pair with the key pair name "test-key" from the creation of a new key pair. -[8] Create an instance

Let's check if EC2 can be started Type the following command

$ cd

$ cd (test-key.PATH to pem)
#If you don't understand the meaning$cd Downloads is OK

$ chmod 400 test-key.pem

$ ssh -i test-key.pem [email protected]
#xxx is the EC2 public IPv4 address

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

#If the above EC2 characters are displayed, the startup is complete.

[ec2-user] $ exit
#End

You have now started EC2.

[1.3] Creating RDS

** Creation procedure **

-[1] Creating a database -[2] Engine type: MySQL -[3] Template: Free tier -[4] DB instance identifier: database-test -[5] Master Username: Any name you like -[6] Master Password: Password of your choice -[7] Connected VPC: VPCtest -[8] Tap additional connection settings -[9] VPC Security Group: Newly created -[10] VPC security group name: dbtest-SG -[11] Creating a database

[1.3.2] Security group editing

The amount of learning about security groups is still small, and security may be a loose setting. Try using it in ** test environment only **.

As an image, do you feel that there are 65535 (port number) entrances at home etc. and only the entrances permitted by the security group are unlocked? --HTTP communication uses port 80 --Port 443 is used for HTTPS communication There are typical port numbers such as.

Edit the security group webtest-sg

** Creation procedure **

-[1] Edit the inbound rule of "webtest-sg" from the security group -[2] Add two rules -[3] One is of type "HTTP" and source is "0.0.0.0/0" -[4] The other is type "MYSQL / Aurora", source "0.0.0.0/0" and save the rule -[5] Edit the inbound rule of "dbtest-SG" -[6] Type "MYSQL / Aurora", source "0.0.0.0/0" and save the rule

(I forgot to unify sg to lowercase (； ´Д ｀))

Let's check if RDS can be started Type the following command

$ cd

$ cd (test-key.PATH to pem)

$ ssh -i test-key.pem [email protected]

[ec2-user] $ sudo yum install mysql -y

[ec2-user] $ mysql -h DB endpoint-u DB master user name-p
#You will be asked for a password, so enter it(Characters are not displayed but are typed)(Copy and paste is possible)

>>> Welcome to the MariaDB monitor.
#Is displayed, you can connect to MySQL on RDS.

#Display the list in the database(Lowercase letters are acceptable show database;)
MySQL > SHOW databases;

#Create a database named "dbtest"
MySQL > CREATE DATABASE dbtest;

#End
MySQL > exit

This completes the RDS settings.

[2] Django environment construction

[2.1] Preparation for installing Python3

#update yum
[ec2-user] $ sudo yum update -y

#Install git with yum
[ec2-user] $ sudo yum install git -y

#Next, clone pyenv from the github repository
[ec2-user] $ git clone https://github.com/yyuu/pyenv.git ~/.pyenv

#Allows commands to be typed through the path
[ec2-user] $ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
[ec2-user] $ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
[ec2-user] $ echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
[ec2-user] $ source ~/.bash_profile

#######You can copy and paste all at once
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
source ~/.bash_profile
#################

#Check if pyenv is included
[ec2-user] $ pyenv -v
>>> pyenv 1.2.20-7-gdd62b0d1

[2.1.2] Installing Python3

Check the current Python version Python 2 series ended service on January 1, 2020 For details on Python 2 system, see this article

#Check Python version
[ec2-user] $ python --version
>>> Python 2.7.18

#Migrate to Python 3

#Installation of required dependencies
[ec2-user] $ sudo yum install gcc zlib-devel bzip2 bzip2-devel readline readline-devel sqlite sqlite-devel openssl openssl-devel -y
>>>(Abbreviation)Has completed!

[ec2-user] $ sudo yum install libffi-devel -y

#Check the version of Python currently available for download
[ec2-user] $ pyenv install -l
>>>(Abbreviation) 
    3.8.5　
    3.8.6
   (Abbreviation)


#At the moment(R2,9/26)Latest version in 3.8.Install 6
[ec2-user] $ pyenv install 3.8.6
>>> Downloading Python-3.8.6.tar.xz...
    -> https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz
    Installing Python-3.8.6...
#It will take some time
    Installed Python-3.8.6 to /home/ec2-user/.pyenv/versions/3.8.6


#Switch from Python version 2 to 3
[ec2-user] $ pyenv global 3.8.6
[ec2-user] $ pyenv rehash

[ec2-user] $ python --version
>>> Python 3.8.6

#Packages that will be needed later
[ec2-user] $ pip install --upgrade pip
[ec2-user] $ sudo yum install python-devel mysql-devel -y
[ec2-user] $ pip install mysqlclient

#This completes the migration to Python3

[2.2] Django environment construction

pip is originally included in EC2 (Linux2)

#Check pip version
[ec2-user] $ pip -V
>>> pip 20.2.1 from /home/ec2-user/.pyenv/versions/3.8.6/lib/python3.8/site-packages/pip (python 3.8)

#Install Django
[ec2-user] $ pip install django

#If you get the following error, single quotes according to the content ('xxxx'Please copy and paste inside) and install Django again
>>> WARNING: You are using pip version 20.2.1; however, version 20.2.3 is available.
You should consider upgrading via the 'xxxx' command.

#Django version check
[ec2-user] $ python -m django --version

#Create a project named "test"
[ec2-user] $ django-admin startproject testDjango

If you check it, it should be generated
[ec2-user] $ ls

>>> mysite/
        manage.py
        mysite/
            __init__.py
            settings.py
            urls.py
            asgi.py
            wsgi.py

--Top mysite: You can create it with any name in the root directory. Can be changed. --manage.py ... A command line utility for performing various operations on your Django project. --mysite ・ ・ ・ Package of this project. --mysite / init.py ・ ・ ・ An empty file that this directory is python. --mysite / settings.py ・ ・ ・ Project settings file. --mysite / urls.py ・ ・ ・ Declare the URL. --mysite / asgi.py ・ ・ ・ Entry point of ASGI compatible WEB server that provides the project. --mysite / wsgi.py ・ ・ ・ Entry point of WSGI compatible WEB server for serving the project.

[2.3] Download FileZilla

You can also edit using vim directly in SSH to edit the code of testDjango, but this time I will try using FailZila For super beginners! How to use FileZilla

Download FileZilla for Win FileZilla Download for Mac

Install and open, press the server button on the upper left and the following screen will appear

Setup steps

-[1] Select "SFTP" as the protocol (because FTP does not encrypt communication) -[2] Host enters "EC2 open IP address" -[3] Logon type is "key file" -[4] User "Name when logging in to EC2" -[5] Key file "Key PATH when logging in to EC2" -[6] Port can be blank -[7] Connection

After the setting is completed, the above screen will appear ** Left is local site (in my PC) ** ** Right is remote site (in EC2) ** Files can be passed and rewritten here. Duplicate the testDjango I made for the time being to the local site

After right-clicking the file you want to edit, you can edit it by displaying and editing. Allow editing with PyCharm in the next [2.4]

[2.4] Download PyCharm

The strongest Python integrated development environment PyCharm

How to associate PyCharm

-[1] Right-click the .py file from Finder -[2] Set Open in this application to PyCharm -[3] Change everything

[2.5] Django settings

[ec2-user] $ cd
[ec2-user] $ cd testDjango
[ec2-user] $ python manage.py startapp polls

settings.py

...
(Abbreviation)
...

# DATABASES = {
#     'default': {
#         'ENGINE': 'django.db.backends.sqlite3',
#         'NAME': BASE_DIR / 'db.sqlite3',
#     }
# }

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'dbtest',
        'USER': 'DB master user',
        'PASSWORD': 'DB master user password',
        'HOST': 'DB endpoint',
        'PORT': '3306',
    }
}

...
(Abbreviation)
...

#LANGUAGE_CODE = 'en-us'
LANGUAGE_CODE = 'ja'

# TIME_ZONE = 'UTC'
TIME_ZONE = 'Asia/Tokyo'

Upload with FileZilla

[ec2-user] $ cd
[ec2-user] $ cd testDjango
[ec2-user] $ python manage.py migrate
#If an error occurs here, the possible causes are described below.

[ec2-user] $ python manage.py dbshell

** Cause of error **

--Did you open FileZilla and upload it? --Are there any commands you forgot to type? --Are there any download failure commands left with error? --Is it 3 or higher in python --version? -Did you keep the order of typing the commands in [2.1.2]? --In setting.py, NAME is a DB name created by MySQL (** not a DB identifier **)

** Other ** No module named'_ctypes' occurs when installing pipenv CentOS7.2 + MySQL5.6 + Python3--- pip install mysqlclient gives mysql_config not found

If you feel that it is impossible, you can recreate the instance from scratch or With the EC2 instance selected, press the action and launch a similar one An instance with the same settings will be created.

This completes the EC2, RDS, and Django settings.

in conclusion

I was too bad at naming this time. (´︵`) For example, if the blog name is xblog xblog-vpc xblog-public-subnet xblog-ec2 xblog-rtb xblog-ec2-sg xblog-db-sg I think that it is easier to understand if you do it.

I hope this article helps someone.

Next time I will try to implement the API for this.

Reference site

The following and the sites that I have referred to very much in the middle of this article are introduced. Thank you very much.

[1] Build Python3 environment on EC2 server [2] WEB application development using django-development 1- [3] Up to Django, syncdb in AWS EC2 + RDS environment [4] Connect to Amazon RDS from Django on EC2 [5] Until you publish your Django application (+ MySQL) on AWS EC2 (+ RDS (+ S3)) [6] (Django memo) Set up MySQL in database [7] Create a database with MySQL [8] What to do if you get a SQLite3 error when starting the development server on Django 2.2 [9] Practical introduction for Vim beginners [10] [What is an AWS route table? Let's briefly explain based on VPC and subnet](https://qiita.com/chro96/items/21863e0960ba4ac72470#:~:text=%E6%9C%AC%E9%A1%8C%E3%81%AEAWS % E3% 81% AE% E3% 83% AB% E3% 83% BC% E3% 83% 88% E3% 83% 86% E3% 83% BC% E3% 83% 96% E3% 83% AB% E3 % 81% A8% E3% 81% AF% EF% BC% 9F & text =% E3% 82% B5% E3% 83% 96% E3% 83% 8D% E3% 83% 83% E3% 83% 88% E5% 86% 85% E3% 81% AB% E3% 81% 82% E3% 82% 8B% E3% 82% A4% E3% 83% B3% E3% 82% B9% E3% 82% BF% E3% 83% B3% E3% 82% B9,% E9% 80% 81% E3% 82% 89% E3% 81% AA% E3% 81% 84% E3% 81% AE% E3% 81% A7% E3% 80% 81 % E9% 80% 9A% E4% BF% A1% E3% 81% A7% E3% 81% 8D% E3% 81% BE% E3% 81% 9B% E3% 82% 93% E3% 80% 82)