[PYTHON] Build a TensorFlow development environment on Amazon EC2 with command copy and paste

Introduction

This is a memo for missionary work. It is for people who want to separate the environment because it will be hard for the PC to execute local resources. The $ notation at the beginning is removed for learning copy. We emphasize the effort of rebuilding (it is not on Docker, but you can easily crush the instance even if the construction fails) and the price. If you want to focus on execution speed and rotate the GPU, please refer to Separate article.

procedure

Add Amazon EC2 instance

• https://ap-northeast-1.console.aws.amazon.com/ec2/v2/home?region=ap-northeast-1#LaunchInstanceWizard
• Select Ubuntu package.
• Open the port nicely in the "security group". Details are omitted.
• ssh: 22
• Jupyter: 8888
• If you do not have an AWS contract, use t2.micro (probably) for free for one year from the new contract.
• After creating, copy the public DNS (ec2-xxxxxxxxxx.ap-northeast-1.compute.amazonaws.com) and paste it into a text editor.

ssh login

chmod 600 xxxxxxxxxx.pem
ssh -v -i xxxxxxxxxx.pem [email protected]

Package update

sudo apt-get update && sudo apt-get -y upgrade

Python preparation

pip installation

sudo apt-get install -y python-pip python-dev
sudo pip install -U pip

Machine learning library + TensorFlow

Install frequently used libraries

You have the following installed:

1. numpy: Matrix calculation library
2. scipy: Scientific calculation library
3. pandas: data frame library
4. matplotlib: Graph drawing library
5. Pillow: Image processing library
6. scikit-learn: Machine learning library
7. google-api-python-client: Google API Client for Python

sudo apt-get install -y libpq-dev python-matplotlib gfortran
sudo pip install --upgrade pip
sudo pip install numpy scipy pandas matplotlib Pillow scikit-learn
sudo pip install google-api-python-client

TensorFlow installation

Install the latest version as of 09/01/2016. Please check Latest version before installing.

sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0rc0-cp27-none-linux_x86_64.whl

Jupyter

Jupyter installation

sudo pip install jupyter

Jupyter config change (port setting etc.)

jupyter notebook --generate-config && ipython profile create
echo "c = get_config()" >> /home/ubuntu/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.ip = '*'" >> /home/ubuntu/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.open_browser = False" >> /home/ubuntu/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.port = 8888" >> /home/ubuntu/.jupyter/jupyter_notebook_config.py
echo "c.InteractiveShellApp.matplotlib = 'inline'" >> /home/ubuntu/.ipython/profile_default/ipython_kernel_config.py

Jupyter login password setting

Register your login password. Make a copy of sha1 displayed after registration is completed.

python -c 'import IPython;print(IPython.lib.passwd())'

Add the copied sha1 to config.

echo "c.NotebookApp.password = 'sha1:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'" >> /home/ubuntu/.jupyter/jupyter_notebook_config.py

Start Jupyter

jupyter notebook You can log in at http://ec2-x-x-x-x.ap-northeast-1.compute.amazonaws.com:8888/ when you start with. If you want to start the EC2 instance at the same time, register the start script in /etc/rc.local.

Jupyter operation check

Log in from your browser and see 42 in TensorFlow Hello, World Sample OK

import tensorflow as tf

hello = tf.constant("Hello, TensorFlow!")
sess = tf.Session()
print(sess.run(hello))

a = tf.constant(10)
b = tf.constant(32)
print(sess.run(a + b))