[PYTHON] TensorFlow Tutorial MNIST For ML Beginners

Since TensorFlow has been released, I decided to study neural networks while using it. I will keep a record of my studies depending on my mood.

Try to install

With pip, the CPU version just hits the command as instructed in the readme. I don't have a GPU at home anyway. https://github.com/tensorflow/tensorflow

I was able to confirm that it works properly on Ubuntu and Mac. Windows people don't know how to do it, so please do your best. Once installed, type ʻimport tensorflow as tf` to make sure you don't get angry.

Try to move

I started with the following tutorial. http://tensorflow.org/tutorials/mnist/beginners/index.md

Well, it's like logistic regression with 10-dimensional output, and it's the usual MNIST handwritten character classification. I thought I'd write a diagram or formula, but the tutorial diagram was so beautiful that I thought it was okay.

Please note that you need input_data.py to run the tutorial code.

# -*- coding: utf-8 -*-

import input_data
import tensorflow as tf

#Download and load MNIST dataset
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

#Prepare variables that represent weights and thresholds(The initial value is zero)
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

#Variables for inserting feature vectors during training
x = tf.placeholder("float", [None, 784])

#Define Softmax function
y = tf.nn.softmax(tf.matmul(x, W) + b)

#Variable for entering the true label value during training
y_ = tf.placeholder("float", [None,10])

#Define the loss function with cross entropy
cross_entropy = -tf.reduce_sum(y_ * tf.log(y))

#Define learning method(Step size 0.Aim to minimize cross entropy with the 01 gradient method)
train_step = tf.train.GradientDescentOptimizer(0.005).minimize(cross_entropy)

#Prepare a session
sess = tf.Session()

#Variable initialization process
init = tf.initialize_all_variables()
sess.run(init)

for i in range(1000):
    #Data for use in mini batch
    batch_xs, batch_ys = mnist.train.next_batch(100)
    #Update using gradient
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

#Define a function that returns the correct answer rate
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

#View the results
print sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})

Try to read

Data preparation

#Download and load MNIST dataset
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

Prepare the data using the mysterious module for downloading the MNIST dataset prepared by Google. I don't need this separately, but it is troublesome to rewrite it, so let's use it. Since my home is an ADSL line, it will take some time to download, but it will be okay for your home.

Preparation of variables to be updated

#Prepare variables that represent weights and thresholds(The initial value is zero)
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

Prepare a matrix W representing the weights and a threshold b. It's like an array of numpy, but we prepare it as a type called Variable for handling in TensorFlow. It's troublesome, but let's put up with it. It seems that you can also convert numpy's array to Variable as below.

W = tf.Variable(np.random.uniform(-1, 1, size=[784, 10]))

Prepare variables and functions

#Variables for inserting feature vectors during training
x = tf.placeholder("float", [None, 784])
#Define Softmax function
y = tf.nn.softmax(tf.matmul(x, W) + b)
#Variable for entering the true label value during training
y_ = tf.placeholder("float", [None,10])
#Define the loss function with cross entropy
cross_entropy = -tf.reduce_sum(y_ * tf.log(y))

A mysterious thing called placeholder came out. This is a variable for which no value has been given yet, and when the value of x or y_ is specified later, the operation result using it y You can evaluate what happens to the values of and cross_entropy. Specifically, we evaluate the loss function cross_entropy when we put the feature matrix in x and the true label in y.

Selection of optimization method

#Define learning method(Step size 0.Aim to minimize cross entropy with the 01 gradient method)
train_step = tf.train.GradientDescentOptimizer(0.005).minimize(cross_entropy)

Specify the optimization method and the value you want to minimize. Here we specify the steepest descent method. 0.005 is the step size. It doesn't matter, but the neural network calls it the learning rate, whereas the optimization Since people use the gradient method other than machine learning, it is often called step size.

Creating a session

#Prepare a session
sess = tf.Session()
#Variable initialization process
init = tf.initialize_all_variables()
sess.run(init)

At this point, the mysterious concept of a session has appeared. I'm not sure, but TensorFlow seems to manage variables etc. in this session unit. If you do not create a session and perform initialization processing, Variable etc. so far I can't bring in the value of what I created in. After the initialization process is finished, for example, look at the contents of W via a session as follows.

>>> sess.run(W)
array([[ 0.6923129 , -0.20792764,  0.03128824, ...,  0.91015261,
         0.84531021, -0.81436723],
       [-0.6045441 ,  0.18968499, -0.48082295, ..., -0.65939605,
         0.61858588, -0.2352511 ],
       [-0.56046396, -0.35212722, -0.44472805, ...,  0.82507199,
         0.77793002, -0.87778318],
       ..., 
       [ 0.73705292,  0.13759996, -0.33590671, ...,  0.15150025,
        -0.2162281 , -0.36046752],
       [-0.90121216, -0.09728234, -0.40505442, ...,  0.02105984,
        -0.46720058, -0.49198067],
       [ 0.29820383,  0.80599529,  0.97673845, ..., -0.43288365,
        -0.73505884, -0.8707968 ]], dtype=float32)

learn

for i in range(1000):
    #Data for use in mini batch
    batch_xs, batch_ys = mnist.train.next_batch(100)
    #Update using gradient
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

I put the data in x, y_, which had no specific value, and updated W, b using the specified learning method. This procedure, of course, is also done through the session. I will do it.

remaining

#Define a function that returns the correct answer rate
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
#View the results
print sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})

I'm looking at the correct answer rate.

Impressions

It's quite similar to Theano, but I think it's a little easier to attach to than Theano. (Theano * 2 + Chainer) / 3 impression. TensorBoard looks amazing, so I want to touch it.

Recommended Posts

TensorFlow Tutorial MNIST For ML Beginners
TensorFlow Tutorial -MNIST For ML Beginners
Conducting the TensorFlow MNIST For ML Beginners Tutorial
TensorFlow MNIST For ML Beginners Translation
[Explanation for beginners] TensorFlow tutorial Deep MNIST
Supplementary notes for TensorFlow MNIST For ML Beginners
[Roughly translate TensorFlow Tutorial into Japanese] 1. MNIST For ML Beginners
I tried the MNIST tutorial for beginners of tensorflow.
I tried a TensorFlow tutorial (MNIST for beginners) on Cloud9-Classification of handwritten images-
[Roughly translate TensorFlow Tutorial into Japanese] 2. Deep MNIST For Experts
TensorFlow tutorial tutorial
[Deprecated] Chainer v1.24.0 Tutorial for beginners
TensorFlow Deep MNIST for Experts Translation
I tried running the TensorFlow tutorial with comments (_TensorFlow_2_0_Introduction for beginners)
I tried the TensorFlow tutorial MNIST 3rd
Django tutorial summary for beginners by beginners ③ (View)
Beginners read "Introduction to TensorFlow 2.0 for Experts"
Django tutorial summary for beginners by beginners ⑤ (test)
Roadmap for beginners
Mathematics for ML
[Explanation for beginners] TensorFlow basic syntax and concept
Django tutorial summary for beginners by beginners ⑦ (Customize Admin)
Django tutorial summary for beginners by beginners ⑥ (static file)
Django Tutorial Summary for Beginners by Beginners (Model, Admin)
Django tutorial summary for beginners by beginners ① (project creation ~)
Django tutorial summary for beginners by beginners ④ (Generic View)
Installing TensorFlow on Windows Easy for Python beginners
Code for TensorFlow MNIST Begginer / Expert with Japanese comments
[Translation] NumPy Official Tutorial "NumPy: the absolute basics for beginners"
Spacemacs settings (for beginners)
python textbook for beginners
MNIST image generation program creation by DCGAN (tensorflow tutorial)
Enable GPU for tensorflow
Dijkstra algorithm for beginners
OpenCV for Python beginners
[Explanation for beginners] Introduction to convolution processing (explained in TensorFlow)
How to learn TensorFlow for liberal arts and Python beginners
[For beginners] I tried using the Tensorflow Object Detection API
Installation notes for TensorFlow for Windows
Learning flow for Python beginners
[For beginners] kaggle exercise (merucari)
Linux distribution recommended for beginners
CNN (1) for image classification (for beginners)
Python3 environment construction (for beginners)
Python #function 2 for super beginners
Seaborn basics for beginners ④ pairplot
Basic Python grammar for beginners
100 Pandas knocks for Python beginners
Python for super beginners Python #functions 1
Python #list for super beginners
~ Tips for beginners to Python ③ ~
[For Kaggle beginners] Titanic (LightGBM)
Reference resource summary (for beginners)
Linux command memorandum [for beginners]
Try TensorFlow MNIST with RNN
Convenient Linux shortcuts (for beginners)