[PYTHON] Infrastructure automation tool Ansible for people who are not good at Ruby

Introduction

I think I should start learning Ruby with Ruby in any way in the world, but I wish I could write the same thing with python. Today, behind the famous infrastructure automation tool chef, an automation tool with python I heard that there is one, so I immediately tried using it. Here, we will introduce the procedure for setting up a remote environment with vagrant and setting up the remote machine with ansible. The host environment is Mac OS X Mavericks.

vagrant installation

I've summarized the vagrant installation earlier, so take a look there.

installation of ansible

Just install with pip.

$ pip install ansible

Preparation

First, prepare a remote machine.

$ mkdir ansible_test
$ cd ansible_test
$ vagrant box add ubuntu12.04_amd64 https://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-amd64-vagrant-disk1.box
$ vagrant init ubuntu12.04_amd64

If you already have a suitable box, you don't have to run the box add command. This time, for various reasons, I will use the amd64 version of ubuntu 12.04. I often introduced it on centos on other sites, and I thought it would be useful to have a site that also describes how to handle it with apt-get, so I did that.

After completing vagrant init, a Vagrantfile will be created in your working directory. Modify the contents so that the network is configured between the host and the guest.

config.vm.network :private_network, ip: "192.168.33.10"

Specifically, just uncomment the above. The IP address remains the default, but you can change it to your liking.

Next, write the settings in ~ / .ssh / config so that you can access without a password.

Host 192.168.33.10
  User vagrant
  IdentityFile /Users/<username>/.vagrant.d/insecure_private_key

Finally, set the hosts file for ansible. This can be the default one, but it is not easy to use, so it is better to make it

[servers]
192.168.33.10  #Specify the IP of the guest OS

Write the above contents to a file called hosts in the ansible-test directory. You can write more than one as the name of servers.

Edit as appropriate for your environment.

This completes the preparation.

Warming up

I don't really need this chapter, but before going to the playbook, I will introduce the basic usage of ansible. This content is the same as other sites, so if you don't like it anymore, you should move on.

First of all, the usual ping confirmation.

$ ansible -i hosts servers -m ping
192.168.33.10 | success >> {
    "changed": false,
    "ping": "pong"
}

Command execution

$ ansible -i hosts servers -a 'pwd'
192.168.33.10 | success | rc=0 >>
/home/vagrant

If you can copy it, go to the next.

Production: Creating a playbook

The essence of this tool is not remote control (remote control can be done with ssh), but automatic infrastructure configuration, so try automatic environment configuration using a yaml file called playbook.

The pages I've seen so far have introduced the method using yum, so the playbook introduced here is ubuntu's apt-get specification (of course, some sites use apt). In order to see if it can be used practically, try to prepare an environment that requires a slightly complicated procedure. This time, I will install Theano, which is a Python package used in Deep Learning.

playbook-theano.yaml


- hosts: servers
  sudo: true
  user: vagrant
  tasks:
    - name: apt-get update
      apt: update_cache=yes
    - name: apt-get upgrade
      apt: upgrade=yes
    - name: apt-get dist-upgrade
      apt: upgrade=dist
    - name: apt-get install git make python-dev python-setuptools libblas-dev gfortran g++ python-pip python-numpy python-scipy liblapack-dev
      apt: name={{ item }} state=latest
      with_items:
        - git
        - make
        - python-dev
        - python-setuptools
        - libblas-dev
        - gfortran
        - g++
        - python-pip
        - python-numpy
        - python-scipy
        - liblapack-dev
    - name: pip install nose
      pip: name=nose
    - name: pip install theano
      command: /usr/bin/pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
    - name: get deb package from cuda, http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb
      get_url: url="http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb" dest="/home/vagrant"
    - name: dpkg install cuda repo
      command: dpkg -i /home/vagrant/cuda-repo-ubuntu1204_6.0-37_amd64.deb
    - name: apt-get update
      apt: update_cache=yes
    - name: apt-get install cuda
      apt: name=cuda state=latest
    - name: add cuda path to environmental variable (1/2)
      lineinfile: dest=/home/vagrant/.bashrc state=present line="export PATH=$PATH:/usr/local/cuda-6.0/bin:" insertafter=EOF
    - name: add cuda path to environmental variable (2/2)
      lineinfile: dest=/home/vagrant/.bashrc state=present line="export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-6.0/lib64:" insertafter=EOF

First, check if this yaml is grammatically incorrect.

$ ansible-playbook -i hosts playbook_theano.yaml --syntax-check

If not wrong, execute.

$ ansible-playbook -i hosts playbook_theano.yaml

It is not necessary to specify the server this time because it is specified in the yaml file, and then each command is executed in sequence. The execution result is as follows

PLAY [servers] **************************************************************** 

GATHERING FACTS *************************************************************** 
The authenticity of host '192.168.33.10 (192.168.33.10)' can't be established.
RSA key fingerprint is XXXXXXXXXXXXXX
Are you sure you want to continue connecting (yes/no)? yes
ok: [192.168.33.10]

TASK: [apt-get update] ******************************************************** 
ok: [192.168.33.10]

TASK: [apt-get upgrade] ******************************************************* 
changed: [192.168.33.10]

TASK: [apt-get dist-upgrade] ************************************************** 
ok: [192.168.33.10]

TASK: [apt-get install git make python-dev python-setuptools libblas-dev gfortran g++ python-pip python-numpy python-scipy liblapack-dev] *** 
changed: [192.168.33.10] => (item=git,make,python-dev,python-setuptools,libblas-dev,gfortran,g++,python-pip,python-numpy,python-scipy,liblapack-dev)

TASK: [pip install nose] ****************************************************** 
changed: [192.168.33.10]

TASK: [pip install theano] **************************************************** 
changed: [192.168.33.10]

TASK: [get deb package from cude, http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb] *** 
changed: [192.168.33.10]

TASK: [dpkg install cuda repo] ************************************************ 
changed: [192.168.33.10]

TASK: [apt-get update] ******************************************************** 
ok: [192.168.33.10]

TASK: [apt-get install cuda] ************************************************** 
changed: [192.168.33.10]

TASK: [add cuda path to environmental variable (1/2)] ************************* 
changed: [192.168.33.10]

TASK: [add cuda path to environmental variable (2/2)] ************************* 
changed: [192.168.33.10]

PLAY RECAP ******************************************************************** 
192.168.33.10              : ok=13   changed=9    unreachable=0    failed=0   

Check if it is actually installed.

$ vagrant ssh
vagrant@vagrant-ubuntu-precise-64:~$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> 
vagrant@vagrant-ubuntu-precise-64:~$ 

It looks okay. In addition, in this yaml file, a set of packages for using theano on GPU is also installed, but it can not be used with VirtualBox, so I'm sorry.

In addition, I think that you will have to repeat vagrant destroy and vagrant up several times to check if a clean installation is possible until the build passes properly, but if the IP address used is the same, ssh will throw an error. Don't forget to delete ~ / .ssh / known_hosts (or delete only that part) every time.

Playbook Tips

A brief summary of the task collection used in the playbook

apt-get

There is an apt module.

- name: apt-get update
  apt: update_cache=yes
- name: apt-get upgrade
  apt: upgrade=yes
- name: apt-get dist-upgrade
  apt: upgrade=dist
- name: apt-get install git
  apt: name=git state=latest

From top to bottom, it is a setting example of apt-get update, apt-get upgrade, apt-get dist-upgrade, apt-get install.

If you want to install multiple packages at once, you can include a loop.

- name: apt-get install git make python-dev python-setuptools libblas-dev gfortran g++ python-pip python-numpy python-scipy liblapack-dev
  apt: name={{ item }} state=latest
  with_items:
    - git
    - make
    - python-dev
    - python-setuptools
    - libblas-dev
    - gfortran
    - g++
    - python-pip
    - python-numpy
    - python-scipy
    - liblapack-dev

Get the source from the outside

Sometimes you want to build from source without going through a package management system. In that case, get the source with the get_url module.

- name: get deb package from cuda
  get_url: url="http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb" dest="/home/vagrant"

Command execution

Use the command module if you want more pure command execution

- name: dpkg install cuda repo
  command: dpkg -i /home/vagrant/cuda-repo-ubuntu1204_6.0-37_amd64.deb

pip execution

A module for pip is also available.

- name: pip install nose
  pip: name=nose

However, if you try to do something a little complicated, it will fit, so it may be better to specify it directly with a command.

- name: pip install theano
  command: /usr/bin/pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git

Writing configuration file

Use the lineinfile module to change settings such as PATH

- name: add cuda path to environmental variable (1/2)
  lineinfile: dest=/home/vagrant/.bashrc state=present line="export PATH=$PATH:/usr/local/cuda-6.0/bin:" insertafter=EOF

This time, I just added a line to the end of the file, but it also provides a function to find the specified line with a regular expression and rewrite it.

Multi-line input

Entering multiple lines does not work just by inserting a newline. It is more straightforward to git clone and then copy it to an appropriate location, or get it from gist with wget, but if it is not so much, you can write it in a loop structure as follows.

- name: add wsgi settings
  lineinfile:
    dest=/etc/apache2/httpd.conf
    state=present
    line={{ item }}
    insertafter=EOF
  with_items:
    - "'WSGIScriptAlias / /var/www/hoge/wsgi.py'"
    - "'WSGIPythonPath /var/www/hoge/'"
    - "'<Directory /var/www/hoge/>'"
    - "'<Files wsgi.py>'"
    - "'Order deny,allow'"
    - "'Allow from all'"
    - "'</Files>'"
    - "'</Directory>'"

The above is the setting when adding wsgi of python to httpd.conf of apache2. The point is to use two quotation marks, single quotation mark and double quotation mark, and put quotation marks in the for loop and line.

Finally

Actually, I'm not so happy so far, and this vagrant + ansible environment is effective when combined with the amazon ec2 service. Next time, I will touch on how to do it.

Recommended Posts

Infrastructure automation tool Ansible for people who are not good at Ruby
Developed a simple spreadsheet app for people who are not good at Excel
NumPy example collection for those who are not good at math
Java SE8 Gold measures (for those who are not good at it)
To people who are "recruiting but not recruiting"
[YOLO v5] Object detection for people who are masked and those who are not