I think I should start learning Ruby with Ruby in any way in the world, but I wish I could write the same thing with python. Today, behind the famous infrastructure automation tool chef, an automation tool with python I heard that there is one, so I immediately tried using it. Here, we will introduce the procedure for setting up a remote environment with vagrant and setting up the remote machine with ansible. The host environment is Mac OS X Mavericks.
I've summarized the vagrant installation earlier, so take a look there.
Just install with pip.
$ pip install ansible
First, prepare a remote machine.
$ mkdir ansible_test
$ cd ansible_test
$ vagrant box add ubuntu12.04_amd64 https://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-amd64-vagrant-disk1.box
$ vagrant init ubuntu12.04_amd64
If you already have a suitable box, you don't have to run the box add command. This time, for various reasons, I will use the amd64 version of ubuntu 12.04. I often introduced it on centos on other sites, and I thought it would be useful to have a site that also describes how to handle it with apt-get, so I did that.
After completing vagrant init, a Vagrantfile will be created in your working directory. Modify the contents so that the network is configured between the host and the guest.
config.vm.network :private_network, ip: "192.168.33.10"
Specifically, just uncomment the above. The IP address remains the default, but you can change it to your liking.
Next, write the settings in ~ / .ssh / config so that you can access without a password.
Host 192.168.33.10
User vagrant
IdentityFile /Users/<username>/.vagrant.d/insecure_private_key
Finally, set the hosts file for ansible. This can be the default one, but it is not easy to use, so it is better to make it
[servers]
192.168.33.10 #Specify the IP of the guest OS
Write the above contents to a file called hosts in the ansible-test directory. You can write more than one as the name of servers.
Edit as appropriate for your environment.
This completes the preparation.
I don't really need this chapter, but before going to the playbook, I will introduce the basic usage of ansible. This content is the same as other sites, so if you don't like it anymore, you should move on.
First of all, the usual ping confirmation.
$ ansible -i hosts servers -m ping
192.168.33.10 | success >> {
"changed": false,
"ping": "pong"
}
Command execution
$ ansible -i hosts servers -a 'pwd'
192.168.33.10 | success | rc=0 >>
/home/vagrant
If you can copy it, go to the next.
The essence of this tool is not remote control (remote control can be done with ssh), but automatic infrastructure configuration, so try automatic environment configuration using a yaml file called playbook.
The pages I've seen so far have introduced the method using yum, so the playbook introduced here is ubuntu's apt-get specification (of course, some sites use apt). In order to see if it can be used practically, try to prepare an environment that requires a slightly complicated procedure. This time, I will install Theano, which is a Python package used in Deep Learning.
playbook-theano.yaml
- hosts: servers
sudo: true
user: vagrant
tasks:
- name: apt-get update
apt: update_cache=yes
- name: apt-get upgrade
apt: upgrade=yes
- name: apt-get dist-upgrade
apt: upgrade=dist
- name: apt-get install git make python-dev python-setuptools libblas-dev gfortran g++ python-pip python-numpy python-scipy liblapack-dev
apt: name={{ item }} state=latest
with_items:
- git
- make
- python-dev
- python-setuptools
- libblas-dev
- gfortran
- g++
- python-pip
- python-numpy
- python-scipy
- liblapack-dev
- name: pip install nose
pip: name=nose
- name: pip install theano
command: /usr/bin/pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
- name: get deb package from cuda, http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb
get_url: url="http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb" dest="/home/vagrant"
- name: dpkg install cuda repo
command: dpkg -i /home/vagrant/cuda-repo-ubuntu1204_6.0-37_amd64.deb
- name: apt-get update
apt: update_cache=yes
- name: apt-get install cuda
apt: name=cuda state=latest
- name: add cuda path to environmental variable (1/2)
lineinfile: dest=/home/vagrant/.bashrc state=present line="export PATH=$PATH:/usr/local/cuda-6.0/bin:" insertafter=EOF
- name: add cuda path to environmental variable (2/2)
lineinfile: dest=/home/vagrant/.bashrc state=present line="export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-6.0/lib64:" insertafter=EOF
First, check if this yaml is grammatically incorrect.
$ ansible-playbook -i hosts playbook_theano.yaml --syntax-check
If not wrong, execute.
$ ansible-playbook -i hosts playbook_theano.yaml
It is not necessary to specify the server this time because it is specified in the yaml file, and then each command is executed in sequence. The execution result is as follows
PLAY [servers] ****************************************************************
GATHERING FACTS ***************************************************************
The authenticity of host '192.168.33.10 (192.168.33.10)' can't be established.
RSA key fingerprint is XXXXXXXXXXXXXX
Are you sure you want to continue connecting (yes/no)? yes
ok: [192.168.33.10]
TASK: [apt-get update] ********************************************************
ok: [192.168.33.10]
TASK: [apt-get upgrade] *******************************************************
changed: [192.168.33.10]
TASK: [apt-get dist-upgrade] **************************************************
ok: [192.168.33.10]
TASK: [apt-get install git make python-dev python-setuptools libblas-dev gfortran g++ python-pip python-numpy python-scipy liblapack-dev] ***
changed: [192.168.33.10] => (item=git,make,python-dev,python-setuptools,libblas-dev,gfortran,g++,python-pip,python-numpy,python-scipy,liblapack-dev)
TASK: [pip install nose] ******************************************************
changed: [192.168.33.10]
TASK: [pip install theano] ****************************************************
changed: [192.168.33.10]
TASK: [get deb package from cude, http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb] ***
changed: [192.168.33.10]
TASK: [dpkg install cuda repo] ************************************************
changed: [192.168.33.10]
TASK: [apt-get update] ********************************************************
ok: [192.168.33.10]
TASK: [apt-get install cuda] **************************************************
changed: [192.168.33.10]
TASK: [add cuda path to environmental variable (1/2)] *************************
changed: [192.168.33.10]
TASK: [add cuda path to environmental variable (2/2)] *************************
changed: [192.168.33.10]
PLAY RECAP ********************************************************************
192.168.33.10 : ok=13 changed=9 unreachable=0 failed=0
Check if it is actually installed.
$ vagrant ssh
vagrant@vagrant-ubuntu-precise-64:~$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>>
vagrant@vagrant-ubuntu-precise-64:~$
It looks okay. In addition, in this yaml file, a set of packages for using theano on GPU is also installed, but it can not be used with VirtualBox, so I'm sorry.
In addition, I think that you will have to repeat vagrant destroy and vagrant up several times to check if a clean installation is possible until the build passes properly, but if the IP address used is the same, ssh will throw an error. Don't forget to delete ~ / .ssh / known_hosts (or delete only that part) every time.
Playbook Tips
A brief summary of the task collection used in the playbook
apt-get
There is an apt module.
- name: apt-get update
apt: update_cache=yes
- name: apt-get upgrade
apt: upgrade=yes
- name: apt-get dist-upgrade
apt: upgrade=dist
- name: apt-get install git
apt: name=git state=latest
From top to bottom, it is a setting example of apt-get update, apt-get upgrade, apt-get dist-upgrade, apt-get install.
If you want to install multiple packages at once, you can include a loop.
- name: apt-get install git make python-dev python-setuptools libblas-dev gfortran g++ python-pip python-numpy python-scipy liblapack-dev
apt: name={{ item }} state=latest
with_items:
- git
- make
- python-dev
- python-setuptools
- libblas-dev
- gfortran
- g++
- python-pip
- python-numpy
- python-scipy
- liblapack-dev
Sometimes you want to build from source without going through a package management system. In that case, get the source with the get_url module.
- name: get deb package from cuda
get_url: url="http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb" dest="/home/vagrant"
Use the command module if you want more pure command execution
- name: dpkg install cuda repo
command: dpkg -i /home/vagrant/cuda-repo-ubuntu1204_6.0-37_amd64.deb
A module for pip is also available.
- name: pip install nose
pip: name=nose
However, if you try to do something a little complicated, it will fit, so it may be better to specify it directly with a command.
- name: pip install theano
command: /usr/bin/pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
Use the lineinfile module to change settings such as PATH
- name: add cuda path to environmental variable (1/2)
lineinfile: dest=/home/vagrant/.bashrc state=present line="export PATH=$PATH:/usr/local/cuda-6.0/bin:" insertafter=EOF
This time, I just added a line to the end of the file, but it also provides a function to find the specified line with a regular expression and rewrite it.
Entering multiple lines does not work just by inserting a newline. It is more straightforward to git clone and then copy it to an appropriate location, or get it from gist with wget, but if it is not so much, you can write it in a loop structure as follows.
- name: add wsgi settings
lineinfile:
dest=/etc/apache2/httpd.conf
state=present
line={{ item }}
insertafter=EOF
with_items:
- "'WSGIScriptAlias / /var/www/hoge/wsgi.py'"
- "'WSGIPythonPath /var/www/hoge/'"
- "'<Directory /var/www/hoge/>'"
- "'<Files wsgi.py>'"
- "'Order deny,allow'"
- "'Allow from all'"
- "'</Files>'"
- "'</Directory>'"
The above is the setting when adding wsgi of python to httpd.conf of apache2. The point is to use two quotation marks, single quotation mark and double quotation mark, and put quotation marks in the for loop and line.
Actually, I'm not so happy so far, and this vagrant + ansible environment is effective when combined with the amazon ec2 service. Next time, I will touch on how to do it.
Recommended Posts