I tried automatic deployment with CircleCI + Capistrano + AWS (EC2) + Rails

Introduction

The other day, I deployed my own portfolio on AWS using Capistrano, so I tried to automatically deploy it in combination with CircleCI.

I started learning programming for about 4 months, but I was able to implement it in about a week.

** I want to incorporate CircleCI / CD into my portfolio! I hope it will be helpful for those who say **.

If you have any suggestions, I would appreciate it if you could comment.

Premise

--Rails app created --CIrcleCI has already introduced automated testing --Deployed on AWS using Capistrano

CircleCI and Capistrano are summarized in a separate article.

Run SystemSpec (RSpec) and Rubocop on CircleCI) Easy deployment with Capistrano + AWS (EC2) + Rails

procedure

You need to make multiple settings, so let's do it step by step.

1. Register your ssh private key with CircleCI to access your AWS EC2 instance
2. Set environment variables on the CircleCI console
3. Edit .circleci / config.yml to make sure you have an SSH connection
4. Run the Capistrano deployment when pushed to Github
5. Set the Capistrano deployment to run only when the Github branch is master

1. Register the SSH private key with CircleCI

After selecting the project to use on the CIrcleCI GUI, select "Project settings >> SSH KEYS >> Add SSH key" and enter the Host Name and Private Key.

Host Name describes the domain or IP. I wrote ʻappname.com` because I had my own domain.

Private Key describes the contents of the private key, However, there are two points to note here, so I will explain them. (I got stuck here for a couple of days ...)

Precautions when registering a private key

1. ** Describe the contents of the private key used to log in to EC2 locally **

If you create your own application and deploy it to AWS, you should have multiple private keys such as AWS access key and private key for linking with Github. Therefore, you may be wondering which private key to use. (I was at a loss ...)

The key you need is the ** private key used to log in to EC2 locally **.

If the private key is stored in ~ / .ssh, check the list of private keys with the following command.

(local)

[~]$ cd ~/.ssh
[.ssh]$ ls

In my case, I use the following command when logging in to EC2 locally.

(local)

[~]$ ssh appname_rsa

In that case, you can copy the contents of the private key by typing the following command on the terminal.

(local)

[~]$ pbcopy < ~/.ssh/appname_rsa

Paste the copied contents into Private Key.

And if the beginning of the description is ----- BEGIN RSA PRIVATE KEY -----, it's OK is.

If it is ----- BEGIN OPENSSH PRIVATE KEY -----, proceed to the next note.

2. ** The private key file format must be PEM format **

There are ʻOPENSSH and PEMfile formats for SSH Key, and the file format for SSH Key set in CircleCI is specified asPEM`.

You can tell how to distinguish the file format at the beginning of the contents of the private key. ʻFor OPENSSH: ----- BEGIN OPENSSH PRIVATE KEY ----- ForPEM: ----- BEGIN RSA PRIVATE KEY ----- `

If the private key used to log in to EC2 locally is in the ʻOPENSSH format, you need to create a private key in the PEM` format and set it so that you can log in to EC2.

I created it in ʻOPENSSH` format, so I recreated the private key by following the steps below.

PEM format private key creation / login setting method

First, generate the key locally.

(local)

[~]$ cd .ssh
[.ssh]$ ssh-keygen -m pem
(#Create public key)
-----------------------------
Enter file in which to save the key ():appname_rsa 
(#Enter the name of the file here)
Enter passphrase (empty for no passphrase): 
(#Enter as it is without doing anything)
Enter same passphrase again: 
(#Enter as it is without doing anything)
-----------------------------

[.ssh]$ ls
#「appname_rsa "and" appname_rsa.Confirm that "pub" has been generated
[.ssh]$ cat appname_rsa.pub
(#Output the contents of the key on the terminal → ssh-rsa~~~~Copy up to local)

Next, set the public key created earlier on the server side (EC2).

(server)

[yuki|~]$ mkdir .ssh
[yuki|~]$ chmod 700 .ssh
[yuki|~]$ cd .ssh
[yuki|.ssh]$ vim authorized_keys
(#vim opens)
-----------------------------
ssh-rsa sdfjerijgviodsjcIKJKJSDFJWIRJGIUVSDJFKCNZKXVNJSKDNVMJKNSFUIEJSDFNCJSKDNVJKDSNVJNVJKDSNVJKNXCMXCNMXNVMDSXCKLMKDLSMVKSDLMVKDSLMVKLCA [email protected]
(#Paste the contents of the key you copied earlier)
-----------------------------
[yuki|.ssh]$ chmod 600 authorized_keys
[yuki|.ssh]$ exit
[ec2-user|~]$ exit

When you're done, go back to your locals and set which communications to use for authentication.

(local)

[~]$ cd .ssh
[.ssh]$ vim config
(#Start Vim and edit the configuration file)

#Add the following
Host appname_rsa
Hostname EC2 Elastic IP(#According to your settings)
  Port 22
  User yuki (#EC2 username)
  IdentityFile ~/.ssh/appname_rsa (#Private key setting)
-----------------------------

This enables SSH communication using the PEM format private key. Enter the following command locally and try to log in.

(local)

ssh appname_rsa

If you can log in, the settings are complete.

If you have done so far, refer to Note 1 and describe the contents of the private key in the Private Key of CircleCI and register it.

2. Set environment variables on the CircleCI console

CircleCI deploys based on ** Github source code **. Therefore, files that are not pushed on Github as described in gitignore cannot be recognized.

And CircleCI has a function to manage such files by setting it as an environment variable on the console, so use it.

From the CircleCI project settings, go to the ʻEnvironment Variables page and select ʻAdd Variable.

Then set two environment variables.

Name:'RAILS_MASTER_KEY' Value:Local'master.key'Describe the contents.

Name:'PRODUCTION_SSH_KEY'　Value:'~/.ssh/appname_rsa_xxxxxxxxxxxxxxx~'

After ʻappname_rsa_ of Value of PRODUCTION_SSH_KEY, write the character string without: of Fingerprints described next to the Host Name of SSH Key` registered earlier.

Next, set up Capistrano's SSH connection in the production environment. Here, let's write config / deploy / production.rb using PRODUCTION_SSH_KEY.

config/deploy/production.rb

server 'Describe EC2 Elastic IP', user: 'yuki', roles: %w[app db web]

#SSH connection using environment variables set in CircleCI GUI
set :ssh_options, {
  keys: [ENV.fetch('PRODUCTION_SSH_KEY').to_s],
  forward_agent: true,
  auth_methods: %w[publickey]
}

3. Confirm that SSH communication is possible

Add the following description to .circleci / config.yml. In Fingerprints, it is written next to the Host Name of the SSH Key you registered earlier, so copy it.

yml:.circleci/config.yml

     - add_ssh_keys:
          fingerprints:
            - "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"

And let's push to github.

If it can be executed, in the console on the CircleCI side, for the process called ʻInstalling additional ssh keys`

Installed key XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX

Should be displayed.

4. Run Capistrano when pushing to Github

Write the following in .circleci / config.yml.

yml:.circleci/config.yml

      - add_ssh_keys:
          fingerprints:
            - "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"

      - deploy:
          name: Capistrano deploy
          command: bundle exec cap production deploy

Let's push to Github. You should be able to deploy automatically! ...is.

5. Set the Capistrano deployment to run only when the Github branch is master

Add filters to the end of workflows in circleci / config.yml.

yml:.circleci/config.yml

# build,test,Describe deploy.
・
・
・

workflows:
  version: 2
  build_accept_deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

Now the Capistrano deployment will only run when the Github branch is master!

Summary

I stumbled upon the SSH certification part lol.

I hope it helps those who are similarly stumbling ☺️

Finally, I will post the source code and reference articles of CircleCI and Capistrano for your reference.

Source code

yml:.circleci/config.yml

version: 2.1

orbs:
  ruby: circleci/[email protected]

jobs:
  build:
    docker:
      - image: circleci/ruby:2.5.1-node-browsers
        environment:
          BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps

  test:
    parallelism: 3
    docker:
      - image: circleci/ruby:2.5.1-node-browsers
        environment:
          DB_HOST: 127.0.0.1
          RAILS_ENV: test
          BUNDLER_VERSION: 2.1.4
      - image: circleci/mysql:8.0
        command: --default-authentication-plugin=mysql_native_password
        environment:
          MYSQL_ALLOW_EMPTY_PASSWORD: 'true'
          MYSQL_ROOT_HOST: '%'
    steps:
      - checkout
      - ruby/install-deps
      - run: mv config/database.yml.ci config/database.yml 
      - run:
          name: Wait for DB
          command: dockerize -wait tcp://localhost:3306 -timeout 1m
      - run: bundle exec rake db:create
      - run: bundle exec rake db:schema:load
      # Run rspec in parallel
      - ruby/rspec-test
      - ruby/rubocop-check

  deploy:
    docker:
      - image: circleci/ruby:2.5.1-node-browsers
        environment:
          BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps
      - add_ssh_keys:
          fingerprints: "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"
      - deploy:
          name: Capistrano deploy
          command: bundle exec cap production deploy
  

workflows:
  version: 2
  build_accept_deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

config/deploy.rb

#Fixed version of capistrano
lock '3.14.1'

#Application name to deploy
set :application, 'golfour'

#Git repository to clone
set :repo_url, '[email protected]:xxxxxx/xxxxxx.git'

#The branch to deploy. The default does not have to be master.
set :branch, 'master'

#The directory to deploy to.
set :deploy_to, '/var/www/rails/appname'

# secret_base_Added to read the key
set :linked_files, %w[config/master.key]

#A file with a symbolic link.
set :linked_files, fetch(:linked_files, []).push('config/database.yml', 'config/settings.yml', '.env')

#A folder with symbolic links.
set :linked_dirs, fetch(:linked_dirs, []).push('log', 'tmp/pids', 'tmp/cache', 'tmp/sockets', 'vendor/bundle', 'public/system')

#The number of versions to keep. Save up to 5 history.
set :keep_releases, 5

#version of ruby
set :rbenv_ruby, '2.5.1'

#The level of the log to output.
set :log_level, :debug

namespace :deploy do
  desc 'Restart application'
  task :restart do
    invoke 'unicorn:restart'
  end

  desc 'Create database'
  task :db_create do
    on roles(:db) do |_host|
      with rails_env: fetch(:rails_env) do
        within current_path do
          execute :bundle, :exec, :rake, 'db:create'
        end
      end
    end
  end

  desc 'Run seed'
  task :seed do
    on roles(:app) do
      with rails_env: fetch(:rails_env) do
        within current_path do
          execute :bundle, :exec, :rake, 'db:seed'
        end
      end
    end
  end

  after :publishing, :restart

  after :restart, :clear_cache do
    on roles(:web), in: :groups, limit: 3, wait: 10 do
    end
  end
end

config/deploy/production.rb

server 'EC2 Elastic IP', user: 'yuki', roles: %w[app db web]

#SSH connection using environment variables set in CircleCI GUI
set :ssh_options, {
  keys: [ENV.fetch('PRODUCTION_SSH_KEY').to_s],
  forward_agent: true,
  auth_methods: %w[publickey]
}

Reference article

[CircleCI] rails5.2 / Capistrano / Automatic deployment to AWS by CICD environment Automate deployment with CircleCI