[PYTHON] Think about dropouts with MNIST

Let's think about dropout using the article here posted the other day. However, it ends in one word.

What is a dropout?

Overfitting sometimes occurs when deep learning is used for learning. This means that it processes too much training data and becomes a dedicated discriminator, making it impossible to handle other data.

Identification is basically a classification problem, but this classification doesn't work. It's easy to see in the graph, but there are some points, which are not so along the points at first, but eventually a curve close to the points is completed. However, if you learn too much, you will end up with a strange curve, although it is close to each point. You can see that even if you classify it with this, it will not work.

So, after learning to some extent, we will ignore the parts that are not so relevant. This is a "dropout".

In the case of Deep Learning, each node is connected in a hierarchy, but I will delete the node that seems to be unnecessary.

• Instead of disconnecting, the node is deleted.

Description in TensorFlow

In TensorFlow, set the percentage of this "disappearing node" to remain. The function to use is "tensorflow.nn.dropout ()". The ratio is specified by the second argument.

keep_prob = tensorflow.placeholder(tf.float32)
h_fc1_drop = tensorflow.nn.dropout(h_fc1, keep_prob)

In the tutorial, the percentage left when learning is executed is specified as "0.5". (Leave half the nodes and erase half the nodes)

train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

Please do not drop out during testing. (Set "1.0")

accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0})

Value adjustment

There is no choice but to make trial and error as to what the value should be, but it seems that not only the value but also the location and number of dropouts are subject to adjustment.

• I'm sure the time will come when this value, position, and number of times will be adjusted by Deep Learning (distant eyes).