[PYTHON] How to install Fast.ai on Alibaba Cloud GPU and run it on Jupyter notebook
This time, I will show you how to install the fast.ai library on Alibaba Cloud Elastic ** GPU ** Service and how to access it using ** Jupyter ** Notebook.
- This blog is a translation from the English version. You can check the original from here. We use some machine translation. We would appreciate it if you could point out any translation errors. *
Why GPU?
Deep learning in particular requires many matrix multiplications. It has been found that this process can be performed exceptionally well on the GPU (Graphical Processing Unit) compared to the central processing unit (CPU). Prior to the boom in deep learning, GPUs were primarily used for games and other graphics rendering. Today, it is often used for model training and reasoning in deep learning. However, not all GPUs are suitable for this task, and deep learning practitioners typically use NVIDIA GPUs because NVIDIA's CUDA libraries and GPU cards are optimized for deep learning. I will.
Having a GPU itself requires setting up a machine with a GPU and a suitable operating system, so it's not ready to solve deep learning issues. For now, deep learning practitioners are primarily using Ubuntu to handle their experiments. Setting up a local GPU machine box, along with all dependencies, drivers, and software installations, requires time and knowledge in hardware design. Instead, a more efficient approach is to rent a GPU instance through a cloud provider like Alibaba Cloud.
Alibaba Cloud GPU instances are billed according to actual usage, saving you the cost of experimenting with GPUs. Also, don't worry about hardware settings and software installations, as Alibaba Cloud has already done these settings.
Features of Fast.ai
Fast.ai provides the following functions.
- The built-in CUDA library needs to take care of the NVIDIA driver installation, to see how the NVIDIA driver works, enter the following command:
nvidia-smi - Easy installation via
condaorpip. - Out-of-box support for computer vision, natural language processing, tabular data, collaborative filtering models.
The following describes how to install the Fast.ai library on Alibaba Cloud Elastic GPU Service on Ubuntu 16.04 and run it on Jupyter Notebook.
Requirements
- New Alibaba Cloud Ubuntu 16.04 instance with at least 10GB RAM NVIDIA GPU.
- Set Root password on the server
Launch Alibaba Cloud ECS Instance
To purchase an Alibaba Cloud Elastic GPU Service instance, you must first log in to Alibaba Cloud ECS Console. Create a new ECS instance and select Ubuntu 16.04 as the operating system with at least 10GB RAM GPU NVIDIA. Choose an NVIDIA GPU card instead of AMD, as shown in the screenshot below. Some locations do not offer GPU instances, so choose a location close to where you have your GPU instance, or choose the cheapest one.
After selecting the instance type, don't forget to select Ubuntu 16.04 64bit as the OS, check the GPU driver auto-install option and select the latest driver. This option will automatically install the CUDA and GPU drivers when you create the instance.
Then connect to the ECS Instance (https://www.alibabacloud.com/help/ja/doc-detail/25434.htm) and log in as the root user.
Installation of anaconda
The best way to work with the fast.ai library or other libraries using Python is to be able to create that environment through Anaconda and not confuse other things we did on the machine. ..
First, you need to download Anaconda for Linux from this URL: [https://www.anaconda.com/download/#linux](https://www.anaconda.com/download?spm=a2c65.11461447.0. 0.1a843ce9ZkXyNF # linux)
Look at the Python 3.7 download button, right click and copy the link.
After that, paste the link in the terminal and download it with the following command.
curl –o https://repo.anaconda.com/archive/Anaconda3-5.3.0-Linux-x86_64.sh
After the download is complete, verifying the integrity of the data with a SHA-256 checksum cryptographic hash verification is optional, but enter the following command:
sha256-sum Anaconda3-5.3.0-Linux-x86_64.sh
For the output hash, you need to check the appropriate Anaconda version on the Anaconda page. If the hash output is the same, it has been verified.
You can install Anaconda immediately after the download is completed with the following command.
bash Anaconda3-5.3.0-Linux-x86_64.sh
Output:
Welcome to Anaconda3 5.3.0 (by Continuum Analytics, Inc.)
In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
Press Enter to continue the installation process, then press and hold Enter to load the license information. You need to approve the license as shown in Output below.
Output:
Do you approve the license terms? [yes|no]
Enter Yes to approve and start the Anaconda installation.
Now you need to choose where to install Anaconda. If the default installation location is sufficient, press ʻEnter` or specify another location where you want to install Anaconda.
Output:
Anaconda3 will now be installed into this location:
/root/anaconda3
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below
[/root/anaconda3] >>>
The installation process is now started. This process can be time consuming as Anaconda packs hundreds of Python packages. Some of these packages will be used by you throughout the day of machine learning and deep learning learning.
When the installation process is complete, you should see output similar to the following:
Output:
...
installation finished.
Do you wish the installer to prepend the Anaconda3 install location
to PATH in your /root/.bashrc ? [yes|no]
[no] >>>
Entering yes in this step will allow you to use the conda command later.
Output:
Prepending PATH=/root/anaconda3/bin to PATH in /home/sammy/.bashrc
A backup will be made to: /root/.bashrc-anaconda3.bak
...
To start Anaconda, you need to source the bashrc file with the following command.
source ~/.bashrc
After this step, you can verify your Anaconda installation with the following command:
conda list
This command lists all Python packages installed by Anaconda.
In this example, we will create a virtual environment named fast.ai and enter this command.
conda create --name fast.ai python=3
The output of this command displays a list of all installed packages, so enter y if you agree.
When the process is complete, start the environment with the following command to start working.
source activate fast.ai
Output:
(fast.ai) root@ubuntu:~$
When you are done working in the fast.ai environment, close it with the following command.
source deactivate
Install Fast.ai
The Fast.ai library can be found at https://github.com/fastai. The installation procedure is simple and easy. It provides both GPU version and CPU version, but this time I am interested in doing deep learning, so I will install the GPU version fast.ai.
First, enter the following command to install the Pytorch software. Since fast.ai is a software library that runs on Pytorch software, you must first install Pytorch to install fast.ai. Don't forget to enable the fast.ai environment.
source activate fast.ai
conda install -c pytorch pytorch-nightly cuda92
This prints a list of sotoware that conda installs in the fast.ai environment. Enter y if you agree.
Then enter the following command to install the torch vision software.
conda install -c fast.ai torchvision-nightly
When you see the software list output, enter y to agree.
Finally, to install fast.ai, enter the following command:
conda install -c fast.ai fast.ai
When you see the software list output, enter y to agree.
At this stage, we have successfully installed fast.ai and are ready to experiment with deep learning.
Try Fast.ai with Jupyter note example
The best way to test your fast.ai installation is to work with out-of-box examples for computer vision, natural language processing, tabular data, and collaborative filtering models.
Let's use an example of computer vision this time.
Enter the following command to copy the sample data of fast.ai.
git clone https://github.com/fast.ai/fast.ai/tree/master/examples
Now the folder in ~ / examples is
Execute the following command.
jupyter notebook –-ip=0.0.0.0 –allow-root
Doing so will output a running Jupyter Notebook with such a token for first-time access.
http://(your-ip-address)/tree?token=f79ec772e0e53d30a221ecb18d447761e0f5a9d223c20175
Open another terminal and enter the following command:
ssh –L localhost:8888:localhost:8888 root@(your-ip-address)
Enter your password and click Enter.
Open a web browser and paste the above token into the URL address bar.
http://localhost:8888/tree?token=f79ec772e0e53d30a221ecb18d447761e0f5a9d223c22017
This command opens the Jupyter Notebook in the URL, then opens the ~ / examples folder and clicks dogs_cats.ipyb to display the Jupyter Notebook as shown below.
To try the fast.ai library, just type Shift + Enter and run all the commands. You can see how the deep learning process is done. For more information, please visit http://docs.fast.ai/ for all about this software. Please find the material.
- Alibaba Cloud is the No. 1 (2019 Gartner) cloud infrastructure operator in the Asia-Pacific region with two data centers in Japan and more than 60 availability zones in the world. Click here for more information on Alibaba Cloud. Alibaba Cloud Japan Official Page *
Recommended Posts