Introduction

This article

What is Deep Learning? Overview (this post)
Install Caffe, one of the Deep Learning libraries Introduction
Let's do simple learning using Caffe Practice It is composed of three

Although it is also in the title, it is a post about a record left by an amateur who is not a researcher of Deep Learning, so please forgive me for any mistakes and read it. (If there is something wrong, I would appreciate it if you could point it out in the comments)

What is Deep Learning in the first place?

To put it very roughly, there are various viewpoints, but Deep Learning makes Neural Network, which is known as a kind of machine learning, surprisingly multi-layered (Deep), and a large amount of data (images, etc.) that can not be common knowledge until then. It is one of the machine learning methods that makes it possible to learn higher-order concepts such as features and meanings contained in the data using voice).

Why Deep Learning is attracting attention

Deep Learning originated in Paper published by Geoffrey Hinton et al. Of the University of Toronto, Canada in 2006. Around the year, in the [Famous Competition] of image recognition (http://www.image-net.org/), it was a gradual evolution that improved performance by 1-2% a year, but it was ranked second or lower. It came to be noticed by winning the championship with an overwhelming difference in recognition performance of 10% or more.

Recently, Google has said that computers can recognize cats. There have been reports that the UC Berkley team has learned to use Deep Learning to help robots perform simple tasks automatically.

In this way, it is also applied to learning robots, etc., and it is versatile because it has extremely high performance and the machine can automatically learn craftsmanship parts such as image features designed by humans. It is attracting attention because of its high nature, and research is being promoted all over the world.

3 things you need to know to get started with Deep Learning with Caffe

Of course, I haven't had a hard time understanding various things (I'm still at all), but for the time being, amateurs will start deep learning! Before that, I will briefly write only three things you need to know Neural Network Deep Learning is a type of Neural Network. So you need to know about Neural Networks. So what exactly is a Neural Network?

Neural Network is a simplified model of the exchange of electrical signals by neurons in the brain.

In Neural Network, a neuron is modeled as a signal that outputs a signal when a weighted input exceeds a certain threshold value, and multiple neurons are prepared and connected.

Quoted from Wikipedia

In this example, there are 3 neurons in Input, 4 neurons in Hidden, and 2 neurons in Output, Input is connected to Hidden, and Hidden is connected to Output in one direction (there is a neural network that connects to each other, but I will not explain it. ) Furthermore, although not shown in the figure, think that the weight above is on each arrow. Then do the following:

Give an output value (numerical value) to each neuron of Input
For each input and Hidden arrow, calculate the weight of the input * arrow and give it to the connected Hidden neurons.
Each neuron of Hidden adds a geta called a bias value to the sum of the numerical values input in 2. and outputs it if it exceeds the threshold value.
Read Input as Hidden and Hidden as Output and do a few.
Get the output of Output

I will omit the details of the formulas, but these will be finally expressed and calculated as matrix operations. For the time being, I think it would be good to have the image that each neuron receives input from other neurons and outputs it to the next neuron that has exceeded a certain level. By doing this, we were able to model the exchange of electrical signals that propagate in the brain one after another.

In Deep Learning, increase the number of layers (deep = Deep) so that this becomes Input → Hidden → Hidden → ・・・ (many) ・・・ → Hidden → Output.

Then, for example, put the image, that is, the value of each Pixel in Input, and adjust the weight value so that if the image is a cat, the neuron at the top of Output will be 1 and otherwise it will be 0. This is learning on the Neural Network Then, (hopefully) this neuron will fire if it is a cat, this will fire if it is a human, and so on, the Output will respond according to the input image, and the Network will make a superordinate concept cat or human from the image. You will be able to recognize

GPU Neural Network eventually repeats matrix operations and learning progresses Of course, the calculation is done using a computer, but in Deep Learning, as the name "Deep" implies, the matrix operation runs a lot during learning. If the calculation takes a long time, it may take days to get the learning result, which is a problem.

So we use the GPU. The GPU is originally intended to speed up graphics processing, which the CPU cannot do, but the GPU has more cores than the CPU and creates thousands of threads to perform the same processing in parallel. It has the characteristic that it is good at doing it earnestly. On the other hand, matrix operations repeat the sum and product of rows and columns, but they can be calculated independently (the calculation of rows 2 and columns 2 must be waited for the multiplication of rows 1 and columns 1). It's not that you can't do it), so it's very suitable for processing on the GPU. Using the GPU instead of the CPU reduces the time required for final learning, so it is very common in the Deep Learning area to use the GPU for calculations, and it is recommended to use the GPU in the libraries described below.

In particular, nVidia's CUDA library for using the GPU is often used. Please check if it is a CUDA compatible GPU as it is written in the instruction manual of the graphic card with the nVidia chip. List of CUDA compatible chips (I'm sorry if it's not nVidia in the first place. If the CUDA compatible graphic card is cheap, you can buy it from around 20,000 yen!)

By the way, I don't have such a GPU! I have no plans to buy it! Even if you say that, Caffe, which will be described later, can run learning (although it will be slower) even if you use the CPU. Also, if the data is small, learning can be completed in tens of seconds to minutes even with a CPU.

For the time being, I think you should remember that if you use GPU in Deep Learning, learning will finish faster than CPU.

Deep Learning Library

To get started with Deep Leaning, you can code from scratch to model Neurons and build learning mechanisms, but Deep Learning has a handy library to make learning easier. Many are open to the public Here, we will do deep learning using Caffe, which is one of the libraries in it.

Besides Caffe, there are libraries such as Theano and Torch.

There is no clear reason, but the information that it is easier (likely) than Theano, and the library is written in C ++ and can be used from Python, so it seems that the threshold of learning in programming language is low, so here Caffe Proceed with

What I want to say is that there is a library so you can do it without any knowledge.

at the end

Now you are ready to start deep learning In the next article I would like to explain the procedure for installing Caffe on Ubuntu.

Reference material

Deep Learning Caffe Video of Nara Institute of Science and Technology's Deep Learning Lecture (English) Introduction to CUDA Programming

[PYTHON] An amateur tried Deep Learning using Caffe (Overview)