[PYTHON] Does TensorFlow change the image of deep learning? What I thought after touching a little

The other day, Google released the deep learning library TensorFlow. Many people have already tried it, and their impressions and usage have been written. I would like to touch it myself and write what I thought.

When I first read the explanation of deep learning

In the explanation of common deep learning, the following figure is used.

It is introduced as a mechanism reminiscent of the mechanism of nerves. Data is transmitted from nerve cell to nerve cell through synapses, and the nerve cell that receives the data sees the data and sends new data.

I when I understood a little about the mechanism of deep learning

However, when it comes to looking at this figure and reflecting it in the program, I am afraid that many cells must be reproduced. Therefore, it would be easier to understand if you think of nerve cells with the same function as a mass and simplify the figure using a high-dimensional vector space.

Think of nerve cells as synaptic connections in vector space as some kind of mapping (think of it as a function if you don't understand it). This map is often non-linear, but I think many use something that is represented by a composite of a linear map (represented by a matrix) and a simple non-linear function. In machine learning, the performance of this map is improved by learning. If you have a composite of a linear map and a non-linear function, you can represent the part of the linear map with a matrix and update the matrix each time you learn.

Now here

When I was reading the explanation of TensorFlow based on this understanding, I was immediately convinced that I was doing deep learning using graphs, but I sometimes wondered, "What?" That is, although there are many explanations about vertices, there are almost no parts that correspond to edges. In the previous figure, the part of the map, that is, the side of the graph is strengthened by learning, but there is no explanation for that side. Instead, there is Variable, and it seems that you are supposed to learn by updating it.

Apparently, I need to rewrite the image of deep learning within myself.

Note that the mapping part is usually divided into a variable, learning-updated part (a linear mapping part) and an invariant part (a non-linear function part). The variable part is represented by some column of numbers. It can be understood that TensorFlow's style is to treat this as a variable and treat it as a vertex instead of embedding it in an edge. By separating from the edge to the vertex, the data and the function are separated, and the data coming from the input and the training data are treated equally. Then, the image will be as shown in the following figure.

At the time of learning, the side flows backward and the learning data part is updated. If you think about it from such an image, you can think of deep learning as something similar to the mechanism of other applications.