[PYTHON] (Now) Build a GPU Deep Learning environment with GeForce GTX 960

Introduction

I remembered that a GPU (GeForce GTX 960) made by NVIDIA was included in my own desktop PC that I rarely used recently, so I thought about building an environment where deep learning can be done by utilizing this. .. Despite the GPU I bought 5 years ago, the processing speed was extremely fast and I was surprised at how wonderful the GPU was (laughs). msi GeForce GTX960 Gaming 2G MGSV

environment

Things necessary

--NVIDIA GPU

Version confirmation

To use Tensorflow / Keras in a GPU environment, you have to match each of the above requirements with the corresponding version. There is a matching table in Tensorflow site, so be sure to check it. Basically, I think that you should install the corresponding version according to the version of Tensorflow. If for some reason you do not specify the version, the latest version is OK. At the time of posting (2020/03/21), 2.0.0 is the latest, so we will build it with 2.0.0. matching_table.png

Create Conda environment & install Tensorflow-gpu

If the CPU version and GPU version of Tensorflow are mixed, it seems that the CPU version is automatically selected, so create a new environment.

cmd


> conda create -n tf200gpu Python=3.6.4

Once created, enable the environment and install Tensorflow-gpu.

cmd


> conda activate tf200gpu
(tf200gpu)> pip install tensorflow-gpu==2.0.0

↓ If you get this error, refer to the following article

ERROR: cannot uninstall 'wrapt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

-Tensorflow install memorandum -63rd day I installed tensorflow.

After the installation is complete, check the following with pip list. (At this stage, even if you execute import tensorflow in Python, an error will occur because CUDA / cuDNN is not included.)

--Is the CPU version of Tensorflow installed? --Is Tensorflow-gpu installed and version 2.0.0?

Install Keras

I think Keras can be installed without any problems. Version 2.3.1 is included in my environment.

cmd


(tf200gpu)> pip install keras

As with Tensorflow, check if the installation is done with pip list.

Install Visual Studio C ++ 2017

Obtain and install Visual Studio that matches your version of Tensorflow from the Microsoft Download Site (https://visualstudio.microsoft.com/en/vs/older-downloads/). This time it will be 2017, so install Visual Studio Community 2017. Check "Desktop development using C ++ workloads" during installation. visualstudio.png

(It takes quite a while ...)

Install NVIDIA Driver

Select the product from NVIDIA download site and get the installer. For other graphic boards, change accordingly. nvidia_driver.png If you do nothing special and hold down "Next", the installation should be completed without any problems.

CUDA installation

CUDA is a general-purpose parallel computing platform for GPUs developed and provided by NVIDIA. Obtain and install the installer from the CUDA Toolkit Download Site (https://developer.nvidia.com/cuda-toolkit-archive). You need to create a free account to get the installer, so make a mess. This time install CUDA Toolkit 10.0. Select the OS type and version, and select exe (network). (Select local when installing on a PC that cannot connect to the Internet) cudatoolkit.png

After downloading, start the installer and press and hold "Next" like Driver to install.

cuDNN installation

Next, install cuDNN, a library for deep learning published by NVIDA. You will also need an account here, so please log in with the account you created when you were in CUDA. This time, get 7.4.2 for CUDA 10 from cuDNN download site. If you check "I Agree To the Terms of the cuDNN Software License Agreement", you will be given several options. If you don't find the version you want, click "Archived cuDNN Releases" in the red frame to see past releases. cudnn.png When you unzip the downloaded zip, you will find three folders, bin, include, and lib, and a text file called NVIDIA_SLA_cuDNN_Support.txt. Open C: \ Program Files \ NVIDIA GPU Computing Toolkit \ CUDA \ v.10.0 in Windows Explorer, and there are bin, include, lib folders in the same way, so in the downloaded folder Installation is completed by copying and pasting the contents into folders with the same name. You may be asked for administrator privileges, so please allow it.

Check if the GPU is recognized

This completes the installation system. To check if the GPU is recognized properly, try the following at the command prompt.

cmd


(tf200gpu)> python -c "from tensorflow.python.client import device_lib;print(device_lib.list_local_devices())"

It is recognized properly ...! !! checkgpu.png

Let's do it!

Then, let's learn using Tensorflow-gpu ... but it was blocked by the following error.

tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.

When I referred to this article, neither cudatooklit nor cudnn appeared on the conda list ... So, when I tried `conda install cudnn```, candidates came out, so I will install it. By the way, it seems that you can install both cudatoolkit and cudnn with `conda install cudnn```. Installed above:

I ran the learning code again and it worked fine on the GPU! !!

I've installed CUDA and cuDNN twice, so I'm not sure if this is the correct method (and the version is different), but it's working for the time being (laughs). I will update the article if I have any problems in the future or find out the correct way! (If you are familiar with this, please let us know in the comments ~ (´; ω; `))

result

Let's measure with the following Python code for learning made by a certain online training program. This is the learning code of the image recognition program that classifies the input image into three classes using CNN. For the image data for learning, load the one saved in advance in .npy format. I tried learning 50 x 50 pixel images with 157 images (halfway), batch size 32, epoch 100 ...

Processor Time
CPU 0:01:22.239975
GPU 0:00:15.542190

GPU is about 5.4 times faster, isn't it? (By the way, the installed CPU is Intel Core i5-6600K @ 3.5GHz)

train.py


import keras
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.utils import np_utils
import numpy as np
import datetime

classes = ["class1", "class2", "class3"]
num_classes = len(classes)
image_size = 50

def main():
    X_train, X_test, y_train, y_test = np.load("./data.npy", allow_pickle=True)
    X_train = X_train.astype("float") / 256
    X_test = X_test.astype("float") / 256
    y_train = np_utils.to_categorical(y_train, num_classes)
    y_test = np_utils.to_categorical(y_test, num_classes)

    model = model_train(X_train, y_train)
    model_eval(model, X_test, y_test)

def model_train(X, y):
    model = Sequential()
    model.add(Conv2D(32, (3, 3), padding='same', input_shape=X.shape[1:]))
    model.add(Activation('relu'))
    model.add(Conv2D(32, (3, 3)))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))
    
    model.add(Conv2D(64, (3, 3), padding='same'))
    model.add(Activation('relu'))
    model.add(Conv2D(64, (3, 3)))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Flatten())
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout(0.5))
    model.add(Dense(3))
    model.add(Activation('softmax'))

    opt = keras.optimizers.adam(lr=0.0001, decay=1e-6)
    
    model.compile(
        loss='categorical_crossentropy',
        optimizer=opt,
        metrics=['accuracy']
    )

    model.fit(X, y, batch_size=32, epochs=100)

    return model

def model_eval(model, X, y):
    scores = model.evaluate(X, y, verbose=1)
    print('Test Loss: ', scores[0])
    print('Test Accuracy: ', scores[1])

if __name__ == "__main__":
    start_time = datetime.datetime.now()
    main()
    end_time = datetime.datetime.now()
    print("Time: " + str(end_time - start_time))

Recommended Posts

(Now) Build a GPU Deep Learning environment with GeForce GTX 960
Build a Python machine learning environment with a container
Build a machine learning environment
Build a machine learning application development environment with Python
Try to build a deep learning / neural network with scratch
Build a machine learning scikit-learn environment with VirtualBox and Ubuntu
Build a Python execution environment using GPU with GCP Compute engine
Build a Tensorflow environment with Raspberry Pi [2020]
Build a Fast API environment with docker-compose
[Linux] Build a jenkins environment with Docker
Build a python virtual environment with pyenv
Build a modern Python environment with Neovim
Build AI / machine learning environment with Python
[Linux] Build a Docker environment with Amazon Linux 2
Quickly build a python environment for deep learning and data science (Windows)
Build a "Deep learning from scratch" learning environment on Cloud9 (jupyter miniconda python3)
Building a machine learning environment with Tellus GPU server (Sakura high-power computing)
Build a C language development environment with a container
Build a WardPress environment on AWS with pulumi
Build a python environment with ansible on centos6
[Python] Build a Django development environment with Docker
Create a python3 build environment with Sublime Text3
Build a Django environment with Vagrant in 5 minutes
[Memo] Build a virtual environment with Pyenv + anaconda
Build a virtual environment with pyenv and venv
Build a Django development environment with Doker Toolbox
Build a Python environment with OSX El capitan
Quickly build a Python Django environment with IntelliJ
Build a python execution environment with VS Code
Build a python environment to learn the theory and implementation of deep learning
A story about predicting exchange rates with Deep Learning
Build a python virtual environment with virtualenv and virtualenvwrapper
Build a python environment for each directory with pyenv-virtualenv
Build a python virtual environment with virtualenv and virtualenvwrapper
Build a development environment with Poetry Django Docker Pycharm
A scene where GPU is useful for deep learning?
Build a Django environment for Win10 (with virtual space)
Create a machine learning environment from scratch with Winsows 10
Build a machine learning environment natively on Windows 10 (x64)
Build a numerical calculation environment with pyenv and miniconda3
Build a "bot that tells you AV actresses with similar faces" by deep learning
Steps to quickly create a deep learning environment on Mac with TensorFlow and OpenCV
Build a data analysis environment with Kedro + MLflow + Github Actions
Build a python machine learning study environment on macOS sierra
Build a machine learning environment on mac (pyenv, deeplearning, opencv)
Build a Django development environment with Docker! (Docker-compose / Django / postgreSQL / nginx)
Deep Learning Gaiden ~ GPU Programming ~
Build Azure Pipelies with Azure DevOps in a Linux self-hosted environment
Build a Go development environment with VS Code's Remote Containers
Create an environment for "Deep Learning from scratch" with Docker
Build a LAMP environment [CentOS 7]
Deep Kernel Learning with Pyro
Try Deep Learning with FPGA
Build python3 environment with ubuntu 16.04
Build python environment with direnv
[Django] Build a Django container (Docker) development environment quickly with PyCharm
Build a comfortable development environment with VSCode x Remote Development x Pipenv
How to build a python2.7 series development environment with Vagrant
Build a python environment with pyenv (OS X El Capitan 10.11.3)
Build GPU environment with GCP and kaggle official image (docker)
Create a Todo app with Django ① Build an environment with Docker