[LINUX] Deployment procedure on AWS (2) Server (EC2 instance) environment settings

Previous content [** STEP1 Network environment settings ](https://qiita.com/tksh8/items/9a8e88a777a3a4ee7a09#step1-%E3%83%8D%E3%83%83%E3%83%88%E3%83% AF% E3% 83% BC% E3% 82% AF% E7% 92% B0% E5% A2% 83% E8% A8% AD% E5% AE% 9A) [ STEP2 EC2 settings **] (https://qiita.com/tksh8/items/9a8e88a777a3a4ee7a09#step2-ec2%E3%81%AE%E8%A8%AD%E5%AE%9A)

Contents this time [** STEP3 EC2 instance environment construction ] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step3-ec2%E3%82%A4%E3%83%B3%E3%82%B9%E3%82%BF%E3%83%B3%E3%82%B9%E3%81%AE%E7%92%B0%E5%A2%83%E6%A7%8B%E7%AF%89) [ STEP4 Link with git, clone app ] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step4-git%E3%81%A8%E3%81%AE%E9%80%A3%E6%90%BA%E3%82%A2%E3%83%97%E3%83%AA%E3%81%AE%E3%82%AF%E3%83%AD%E3%83%BC%E3%83%B3) [ STEP5 App secret_key_base settings ] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step5-%E3%82%A2%E3%83%97%E3%83%AA%E3%81%AEsecret_key_base%E3%81%AE%E8%A8%AD%E5%AE%9A) [ STEP6 Unicorn settings ] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step6-unicorn%E3%81%AE%E8%A8%AD%E5%AE%9A) [ STEP7 Nginx settings ] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step7-nginx%E3%81%AE%E8%A8%AD%E5%AE%9A) [ STEP8 Migration ] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step8-%E3%83%9E%E3%82%A4%E3%82%B0%E3%83%AC%E3%83%BC%E3%82%B7%E3%83%A7%E3%83%B3) [ STEP9 Start confirmation **] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step9-%E8%B5%B7%E5%8B%95%E7%A2%BA%E8%AA%8D)

STEP3 Build an EC2 instance environment

We will set ruby and other environments to work within the created EC2 instance.

Log in to AWS
[~]$ ssh test_key_rsa

Enter the following
[testuser@ ~]$ sudo yum install git make gcc-c++ patch openssl-devel libyaml-devel libffi-devel libicu-devel libxml2 libxslt libxml2-devel libxslt-devel zlib-devel readline-devel ImageMagick ImageMagick-devel epel-release

You will be asked for a password, so enter it.
Is this ok [y/d/N]: y

Because it is in

Enter with y

Install MySQL 5.7

[testuser@ ~]$ sudo rpm -ivh http://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
[testuser@ ~]$ sudo yum install mysql-devel mysql57 mysql57-server

Install Node.js

[testuser@ ~]$ curl -sL https://rpm.nodesource.com/setup_6.x | sudo bash -
[testuser@ ~]$ sudo yum -y install nodejs

This will also be confirmed, so press "y" → Enter.

Install rbenv

[testuser@ ~]$ git clone git://github.com/sstephenson/rbenv.git ~/.rbenv
[testuser@ ~]$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
[testuser@ ~]$ echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
[testuser@ ~]$ source .bash_profile

-Check the installed rbenv version
[testuser@ ~]$ rbenv --version
rbenv 1.1.2-4-g577f046

Install ruby-build

(#.bash_Load profile)
[testuser@ ~]$ git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
(#ruby-install build)
[testuser@ ~]$ rbenv rehash  

installation of ruby

-Check the installable version
[testuser@ ~]$ rbenv install -l

[testuser@ ~]$ rbenv install 2.5.1
[testuser@ ~]$ rbenv global 2.5.1
[testuser@ ~]$ rbenv rehash


-Check the version of installed Ruby
-If the version of ruby is displayed, it is successful.
[testuser@ ~]$ ruby -v
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]

STEP4 Link with git, clone app

App clone

Clone the created application from git and place it in the EC2 instance

Generate a git configuration file called .gitconfig with the vi command

[testuser@ ~]$ vi .gitconfig

Press "i" to enter edit mode and add the following Please enter your own name and email

[user]
  name = your_name (#My name registered in git)
  email = [email protected] (#My email address when registering with git)

[alias](#This is your favorite)
  a = add
  b = branch
  ch = checkout
  st = status

[color](#coloring)
  ui = true

[url "github:"]
    InsteadOf = https://github.com/
    InsteadOf = [email protected]:

Press "esc" to finish and Save as ": wq"

Save the file when you're done I will create a directory to place the application

[testuser@ ~]$ cd /
[testuser@ ~]$ sudo chown testuser var (#Owner of the var folder as testuser)
[testuser@ ~]$ cd var
[testuser@ var]$ sudo mkdir www
[testuser@ var]$ sudo chown testuser www 
[testuser@ var]$ cd www
(#Same processing as www)
[testuser@ www]$ sudo mkdir rails
[testuser@ www]$ sudo chown testuser rails

Connection with git

First, type in the following command

Create key, key name is "aws_git_rsa", password is blank

[testuser@ www]$ cd ~
[testuser@ ~]$ chmod 700 .ssh
[testuser@ ~]$ cd .ssh
[testuser@ .ssh]$ ssh-keygen -t rsa
-----------------------------
Enter file in which to save the key ():

aws_git_rsa

(#Enter the name of the file here)

Enter passphrase (empty for no passphrase): 
(#Enter as it is without doing anything)

Enter same passphrase again: 
(#Enter as it is without doing anything)
[testuser@ .ssh]$ ls
(#「aws_git_rsa "and" aws "_git_rsa.Confirm that "pub" has been generated)

[[email protected]]$ vi config

Press "i" to enter edit mode and add the following

Host github
  Hostname github.com
  User git
  IdentityFile ~/.ssh/aws_git_rsa

Press "esc" to finish and Save as ": wq"

Check the contents of the key. Copy it somewhere

[[email protected]]$ cat aws_git_rsa.pub

Access GitHub from your browser and select "SSH and GPG keys" in "Settings"

スクリーンショット 2019-10-10 10.41.07.png スクリーンショット 2019-10-10 10.43.25.png

Click "New SSH key", enter "App name --EC2" in Title, "Contents of the key created earlier" in Key, and register with "Add SSH key"

スクリーンショット 2019-10-10 10.43.43.png スクリーンショット 2019-10-10 10.43.50.png

Return to the terminal and change the permissions of the configuration file

[[email protected]]$ chmod 600 config

Confirm connection to GitHub. Yes for questions on the way. Success if the Github user name appears

[[email protected]]$ ssh -T github

#Hi Github username! You've successfully authenticated, but GitHub does not provide shell access.

If such a message is returned, the connection is successful.

After registration, it's time to clone.

[testuser@ .ssh]$ cd /var/www/rails

The URL that follows git clone is The URL of the github of the app I created (ssh's)

[testuser@ .rails]$ git clone [email protected]:~~~~~~~~~~~~

Please execute the following command for confirmation.

[testuser@ .rails]$ ls

If the folder with your app name exists, the clone is successful.

STEP5 Setting the secret_key_base of the app

If the clone is successful, modify the secrets_key_base described in the rails file config / secrets.yml.

Install bundler

[testuser@ .rails]$ cd ~
[testuser@ ~]$ rbenv exec gem install bundler

[testuser@ ~]$ rbenv rehash


[testuser@ ~]$ which bundler
 # /usr/local/rbenv/shims/Success if bundler is output
[testuser@ .rails] cd /var/www/rails/app name

[testuser@app name]$ gem install rubygems-update -v '<3' && update_rubygems
[testuser@app name]$ bundle config --local build.mysql2 "--with-cppflags=-I/usr/local/opt/openssl/include"
[testuser@app name]$ bundle config --local build.mysql2 "--with-ldflags=-L/usr/local/opt/openssl/lib"
[testuser@app name]$ bundle install --path vendor/bundle

[testuser@app name]$ bundle exec rake secret

Copy the displayed key
[testuser@app name]$ vi config/secrets.yml

Press "i" to enter edit mode and add the following

  production:
    secret_key_base: (Paste the one you copied earlier)

Press "esc" to finish and Save as ": wq"

STEP6 Unicorn settings

(Rack HTTP server that provides Ruby web application in UNIX environment (Rails application can be executed on the application server))

[testuser@app name]$ gem install unicorn

Add the following to your Gemfile

[testuser@app name]$ vi Gemfile

Gemfile


group :production, :staging do
  gem 'unicorn', '5.4.1'
end

Press "esc" to finish and Save as ": wq"

[testuser@app name]$ bundle install

[testuser@app name]$ vi config/unicorn.conf.rb

Create config / unicorn.rb and edit the contents as follows

config/unicorn.rb


#Put the directory where the application code on the server is installed in a variable
app_path = File.expand_path('../../', __FILE__)

#Determine application server performance
worker_processes 1

#Specify the directory where the application is installed
working_directory app_path

#Specify the location of the files required to start Unicorn
pid "#{app_path}/tmp/pids/unicorn.pid"

#Specify the port number
listen "#{app_path}/tmp/sockets/unicorn.sock"

#Specify a file to log errors
stderr_path "#{app_path}/log/unicorn.stderr.log"

#Specify the file to record the normal log
stdout_path "#{app_path}/log/unicorn.stdout.log"

#Set maximum time to wait for Rails application response
timeout 60

#The following is an applied setting, so the explanation is omitted.

preload_app true
GC.respond_to?(:copy_on_write_friendly=) && GC.copy_on_write_friendly = true

check_client_connection false

run_once = true

before_fork do |server, worker|
  defined?(ActiveRecord::Base) &&
    ActiveRecord::Base.connection.disconnect!

  if run_once
    run_once = false # prevent from firing again
  end

  old_pid = "#{server.config[:pid]}.oldbin"
  if File.exist?(old_pid) && server.pid != old_pid
    begin
      sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
      Process.kill(sig, File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH => e
      logger.error e
    end
  end
end

after_fork do |_server, _worker|
  defined?(ActiveRecord::Base) && ActiveRecord::Base.establish_connection
end

Press "i" to enter edit mode and add the following

gemfile:unicorn.conf.rb


  # set lets
  $worker  = 2
  $timeout = 30
  $app_dir = "/var/www/rails/mumu" #My application name
  $listen  = File.expand_path 'tmp/sockets/.unicorn.sock', $app_dir
  $pid     = File.expand_path 'tmp/pids/unicorn.pid', $app_dir
  $std_log = File.expand_path 'log/unicorn.log', $app_dir
  # set config
  worker_processes  $worker
  working_directory $app_dir
  stderr_path $std_log
  stdout_path $std_log
  timeout $timeout
  listen  $listen
  pid $pid
  # loading booster
  preload_app true
  # before starting processes
  before_fork do |server, worker|
    defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!
    old_pid = "#{server.config[:pid]}.oldbin"
    if old_pid != server.pid
      begin
        Process.kill "QUIT", File.read(old_pid).to_i
      rescue Errno::ENOENT, Errno::ESRCH
      end
    end
  end
  # after finishing processes
  after_fork do |server, worker|
    defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
  end

Press "esc" to finish and Save as ": wq"

next Comment out the following

config/environments/production.rb


# config.assets.js_compressor = :uglifier

STEP7 Nginx settings

(One of the same web servers as apache etc. However, Nginx and Rack cannot be connected directly, so you need to sandwich Unicorn)

[testuser@ ~]$ cd ~
[testuser@ ~]$ sudo yum install nginx
[testuser@ ~]$ cd /etc/nginx/conf.d/
[testuser@ conf.d]$ sudo vi test.conf #Rename the file with your application name

Press "i" to enter edit mode and add the following

upstream app_server {
  #Settings for linking with Unicorn. Note that the application name is rewritten to your own application name. Probably chat this time-space
  server unix:/var/www/<Application name>/tmp/sockets/unicorn.sock;
}

# {}The part surrounded by is called a block. Can set the server
server {
  #The port number on which this program accepts connections
  listen 80;
  #Request URL to accept connection Cannot be accessed with URL not written here
  server_name <Elastic IP>;

  #Set the maximum size of files uploaded from the client to 2 giga. The default is 1 mega, so keep it large
  client_max_body_size 2g;

#The root directory when the connection came
  root /var/www/<Application name>/public;

#assets file(CSS and JavaScript files, etc.)Settings applied when access comes to
  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @unicorn;

  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://app_server;
  }

  error_page 500 502 503 504 /500.html;
}

The following three points must be rewritten.

Please change the part of on the 3rd line to your own. In the same way, change the part marked on the 11th line to your own. Change the on the 14th line to your own.

Press "esc" to finish and Save as ": wq"

Do the following

[testuser@ conf.d]$ cd /var/lib
[testuser@ lib]$ sudo chmod -R 775 nginx

STEP8 migration

MySQL installation

[testuser@ ~]$ sudo yum install mysql-devel

-Install MySQL Server
[testuser@ ~]$ sudo yum install mysql-server

-MySQL startup settings
[testuser@ ~]$ sudo chkconfig mysqld on

-You can check if the startup setting is successful with the chkconfig command.
-If the following is displayed, the setting is successful.
[testuser@ ~]$ chkconfig
mysqld           0:off   1:off   2:on    3:on    4:on    5:on    6:off

-Start MySQL.
[testuser@ ~]$ sudo vi /etc/my.cnf
Add the following
character-set-server=utf8



[testuser@ ~]$ sudo /etc/init.d/mysqld start

-Or the same below.
[testuser@ ~]$ sudo service mysqld start

-Log in as root and create a new user.
[testuser@ ~]$ mysql -u root
mysql> GRANT ALL PRIVILEGES ON [Database name].* TO [Your app name]@localhost IDENTIFIED BY '[Password to set]';

Check that the character code of MySQL may be latin1 by default.

mysql> SHOW VARIABLES LIKE 'character_set%';
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

- character_set_filesystem and character_sets_It is ok if other than dir is utf8.
-If not, unify to utf8 and restart.

mysql> set character_set_database = utf8;
mysql> set character_set_server = utf8;

Query OK, 0 rows affected (0.00 sec)

mysql> exit

MySQL settings

[testuser@ ~]$ cd /var/www/rails/app name
[testuser@app name]$ vi config/database.yml
----------------------------
  production:
    <<: *default
    database: mumu_production
    username: root #Change here to root
  password:      #Leave this blank
----------------------------


[testuser@app name]$ sudo service mysqld start #Start mysqld
[testuser@app name]$ ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock
[testuser@app name]$ bundle exec rake db:create RAILS_ENV=production
[testuser@app name]$ bundle exec rake db:migrate RAILS_ENV=production

Success if the migration is performed

[** Please proceed to STEP9 startup confirmation **] (https://qiita.com/tksh8/items/3d2f2eeaf0e8de291b6d#step9-%E8%B5%B7%E5%8B%95%E7%A2%BA%E8%AA%8D)

For RDS

Create additional subnets and specify the subnets that belong to the Availability Zone in the following two ・ Ap-northeast-1a ・ Ap-northeast-1c

at top Search for "RDS"

Subnet Group> Click [Create DB Subnet Group]

スクリーンショット 2020-01-08 23.15.21.png
"App name" in the name_DB-Subnet-Group」
"DB Subnet Group for App Name" in the description
"VPC for app name" in VPC ID
so
Click "Add all subnets associated with this VPC" and
[Create]Press

From the dashboard Click [Create Database] on RDS

スクリーンショット 2020-01-08 23.23.14.png
-Easy to create
-MySQL
-Free tier
-"App name" in the DB instance identifier-mysql」
-Master user and master password are arbitrary values
"Create" with
スクリーンショット 2020-01-08 23.37.43.png

The database will be created in a few minutes

Please refer to here

https://qiita.com/nago3/items/1cd6daa5ad6377e325ed

Add and save database information for production as an environment variable of Rails

Rewrite database.yml as below

database.yml


production:
  adapter: mysql2
  encoding: utf8
  reconnect: false
  pool: 5
  host: <%= ENV['DB_HOST'] %>
  username: <%= ENV['DB_USERNAME'] %>
  password: "<%= ENV['DB_PASSWORD'] %>"
  database: <%= ENV['DB_NAME'] %> 

Edit bash_profile

$ sudo vi ~/.bash_profile

Add and save database information for production as an environment variable of Rails

bash_profile


export DB_HOST="RDS endpoint"
export DB_USERNAME="RDS username"
export DB_PASSWORD="RDS password"
export DB_NAME="app name_production"

[testuser@app name]$ source ~/.bash_profile

Check if it is set correctly

[testuser@app name]$ echo $DB_NAME
[testuser@app name]$ echo $DB_USERNAME
[testuser@app name]$ echo $DB_PASSWORD
[testuser@app name]$ echo $DB_HOSTNAME
[testuser@app name]$ echo $DB_PORT
Start mysqld
[testuser@app name]$ sudo service mysqld start

Perform migration
[testuser@app name]$ rake db:create RAILS_ENV=production
[testuser@app name]$ bundle exec rake db:migrate RAILS_ENV=production

STEP9 Start confirmation

SSH into EC2 and precompile your Rails app
[testuser@app name]$ bundle exec rake assets:precompile RAILS_ENV=production


Restart Nginx
[testuser@app name]$ sudo service nginx restart


Launch Unicorn
[testuser@app name]bundle exec unicorn_rails -c config/unicorn.conf.rb -E production -D

Or
$ bundle exec unicorn_rails -E production -c config/unicorn.rb -D

Confirm that Unicon starts
[testuser@app name]$ ps -ef | grep unicorn | grep -v grep
#OK if the list of processes is displayed in 3 steps


#Start unicorn

#Start confirmation
$ ps aux | grep unicorn

#Check the log if unicorn cannot start
$ tailf log/unicorn.log

$ sudo tail -f /var/log/nginx/access.log
$ sudo tail -f /var/log/nginx/error.log 
$ sudo tail -f /var/www/vue_sample/log/unicorn.log


#Start nginx
$ sudo service nginx start
#Start confirmation
$ ps aux | grep nginx

Verification

Access by hitting the IP from the browser * If you do not know the IP address, you can check it from the description of the EC2 instance

Access 52.64. ~~~~~~~~~ (your IP address) Then

http://IPアドレス/ Success if the Rails app works

It will be displayed if it works! !!

Recommended Posts

Deployment procedure on AWS (2) Server (EC2 instance) environment settings
Deployment procedure on AWS ① Network environment settings
# 2 Build a Python environment on AWS EC2 instance (ubuntu18.04)
xgboost (python) on EC2 Spot instance environment prepared by AWS Lambda
# 3 Build a Python (Django) environment on AWS EC2 instance (ubuntu18.04) part2
Create an AWS Cloud9 development environment on your Amazon EC2 instance
Prepare the environment of Chainer on EC2 spot instance with AWS Lambda
code-server online environment (4) Launch code-server on the EC2 instance
Deployment method (2) (EC2 environment construction / Linux / MySQL settings)
[AWS] Install node.js on EC2 instance and execute sample program
Build Keras environment on AWS E2 G2 instance February 2017 version
[AWS EC2] Settings you want to do on Amazon Linux 2
Building an environment to execute python programs on AWS EC2
ruby environment construction with aws EC2
Use jupyter on AWS GPU instance
Building an environment to run ChainerMN on a GPU instance on AWS
Procedure for building a kube environment on amazon linux2 (aws) ~ (with bonus)
I easily built an operating environment for Python3 + Tornado on AWS EC2.
June 2017 version to build Tensorflow / Keras environment on GPU instance of AWS
# 1 Until you deploy Django's web application (instance construction with EC2 on AWS)
Change AWS EC2 instance from t2 to t3
Run TensorFlow on a GPU instance on AWS
AWS EC2 instance launch and ssh connection
Python development server construction procedure (AWS + Anaconda)
[Note] Python environment construction on rental server "CORESERVER"
Build a WardPress environment on AWS with pulumi
[2020 version] How to install Python3 on AWS EC2
Try Tensorflow with a GPU instance on AWS
Build python environment with pyenv on EC2 (ubuntu)
Python development on Ubuntu on AWS EC2 (using JupyterLab)
Creating an AWS EC2 EC2 Instance (Amazon Linux AMI) 2
Security settings required after launching an EC2 instance
code-server online environment (7) Deploy compute on git on EC2
Building a Python environment on a Sakura VPS server