Lua version Deep Learning from scratch Part 1 [Implementation of Perceptron] Lua version Deep Learning from scratch Part 2 [Activation function] Lua version Deep Learning from scratch Part 3 [Implementation of 3-layer neural network] [Lua version Deep Learning from scratch 4 [Implementation of softmax function]] (http://qiita.com/Kazuki-Nakamae/items/20e53a02a8b759583d31) Lua version Deep Learning from scratch Part 5 [Display MNIST image] Lua version Deep Learning from scratch Part 5.5 [Making pkl files available in Lua Torch]
This time, NN inference processing of MNIST image data is performed. In the original document, the pkl file whose weight and bias are determined in advance is read, so this time, the same processing was performed with Torch using this pkl file. For reading pkl files, see Deep Learning from scratch for Lua version 5.5 [Making pkl files available in Lua Torch]. Please refer. The script looks like this:
neuralnet_mnist.lua
require './activationFunc.lua'
require './softmax.lua'
require './exTorch.lua'
require 'image'
npy4th = require 'npy4th' --https://github.com/htwaijry/npy4th (Author:Hani Altwaijry)
---Data acquisition function
--Acquire MNIST data.
-- @return Test data image,Test data label{Type:ByteTensor}
function get_data()
--download
local tar = 'http://torch7.s3-website-us-east-1.amazonaws.com/data/mnist.t7.tgz'
if not paths.dirp('mnist.t7') then
os.execute('wget ' .. tar)
os.execute('tar xvf ' .. paths.basename(tar))
end
--get data
local test_file = 'mnist.t7/test_32x32.t7'
local testData = torch.load(test_file,'ascii')
return testData['data'], testData['labels']
end
---Network generation function.
--Returns a 3-layer NN with determined weights and biases. This time we will use the weights described in pkl
-- @return 3 layer NN(Type：table)
function init_network()
local network = npy4th.loadnpz('sample_weight.npz')
return network
end
---Classification function.
--Classification calculation is performed according to the network.
-- @param network input(Type：torch.table)
-- @param x input
-- @return output(Type：torch.DoubleTensor)
function predict(network, x)
local W1, W2, W3 = network['W1'], network['W2'], network['W3']
local b1, b2, b3 = network['b1'], network['b2'], network['b3']
--The first input makes the tensor one-dimensional to fit the numpy format
local a1 = mulTensor(x:resize(1,W1:size()[1]), W1) + b1:double()
local z1 = sigmoid(a1)
local a2 = mulTensor(z1,W2) + b2:double()
local z2 = sigmoid(a2)
local a3 = mulTensor(z2,W3) + b3:double()
local y = softmax(a3)
return y
end
local x, t = get_data()
local network = init_network()
local accuracy_cnt = 0
for i = 1, x:size()[1] do
scaledx = image.scale(x[i],28,28) --Reduce 32x32 to 28x28
local y = predict(network, scaledx)
local value, indices = torch.max(y, 2) -- torch.max( , 2)Get the maximum value in the row direction with
local p = tensor2scalar(indices) --Since it is a tensor, it is converted to a scalar
if p == t[i] then
accuracy_cnt = accuracy_cnt + 1
end
end
print("Accuracy:"..(accuracy_cnt/x:size()[1]))
In the original document, it was 28x28 MMIST data, but the image used this time is 32x32. Therefore, the reduced data such as image.scale (image data, 28, 28) is subjected to inference processing. Please note that the result will be different from the original. activationFunc.lua, softmax.lua please refer to. exTorch.lua contains auxiliary defined functions to improve readability.
exTorch.lua
---Computational function of the product between tensors
--The product AB of the tensors corresponding to each dimension is performed.
-- @param A A (Type：Tensor)
-- @param B B (Type：Tensor)
-- @return AB (Type：torch.DoubleTensor)
function mulTensor(A, B)
A = A:double()
B = B:double()
local AB = nil;
if (A:dim() == 1 and B:dim() ~= 1) then
--1Dvector / matrix
AB = torch.mv(B:t(), A)
else
--others
AB = A*B
end
return AB
end
---Scalar conversion function for tensor
--Convert 1x1 tensor to scalar.
-- @param tensor 1x1 tensor(Type：Tensor)
-- @return scalar(Type：number)
function tensor2scalar(tensor)
return tensor[1]
end
Execution example
$ th neuralnet_mnist.lua
Accuracy:0.8616
Since it is reduced, it is different from the result in python, but it seems that the inference itself is done properly. This time, there are many processes that are forced to match the python data, but without it, inference processing can be done with the same effort as the original. The only thing to note is that type handling may be more careful than python. In python, implicit type conversion is used instead, but Lua's handling of types is rather standard, so if it is different, an error will be returned. If you know that the type is an integer calculation, use an integer type such as byte or long, and if you are calculating with decimals, use the double type. Although it is basic, it is better to avoid float type as much as possible because there is always an error in computer decimal calculation. If you don't know, try the code below.
cancellation.lua
local floatT = torch.Tensor(1):fill(0):float()
for i = 1, 1000000 do
floatT = floatT + torch.Tensor(1):fill(0.000001):float()
end
local doubleT = torch.Tensor(1):fill(0):double()
for i = 1, 1000000 do
doubleT = doubleT + torch.Tensor(1):fill(0.000001):double()
end
print("float : ")
print(floatT)
print("double : ")
print(doubleT)
Execution example
$ th cancellation.lua
float :
1.0090
[torch.FloatTensor of size 1]
double :
1.0000
[torch.DoubleTensor of size 1]
Originally, the correct calculation is that both are 1, but the result will be different due to an error in float. Double also causes such an error, but it is safer than float.
This time is over.
It feels uncomfortable to do it with the reduced result, so let's extract the MMIST data from python as well. Copy the dataset directory from the files in the original appendix to the current directory, and execute the following script.
saveMMISTtest.py
# coding: utf-8
import sys, os
import numpy as np
import pickle
from dataset.mnist import load_mnist
(x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label=False)
np.save("x_test",x_test)
np.save("t_test",t_test)
Execution example
$ python3 saveMMISTtest.py
Since x_test.npy and t_test.npy are output, read them with get_data ().
neuralnet_mnist2.lua
require './activationFunc.lua'
require './softmax.lua'
require './exTorch.lua'
npy4th = require 'npy4th' --https://github.com/htwaijry/npy4th (Author:Hani Altwaijry)
---Data acquisition function
--Acquire MNIST data.
-- @return Test data image,Test data label{Type:ByteTensor}
function get_data()
--image data
local x_test = npy4th.loadnpy('x_test.npy')
--Label data
local t_test = npy4th.loadnpy('t_test.npy')
return x_test, t_test
end
---Network generation function.
--Returns a 3-layer NN with determined weights and biases. This time we will use the weights described in pkl
-- @return 3 layer NN(Type：table)
function init_network()
local network = npy4th.loadnpz('sample_weight.npz')
return network
end
---Classification function.
--Classification calculation is performed according to the network.
-- @param network input(Type：torch.table)
-- @param x input
-- @return output(Type：torch.DoubleTensor)
function predict(network, x)
local W1, W2, W3 = network['W1'], network['W2'], network['W3']
local b1, b2, b3 = network['b1'], network['b2'], network['b3']
--The first input makes the tensor one-dimensional to fit the numpy format
local a1 = mulTensor(x, W1) + b1:double()
local z1 = sigmoid(a1)
local a2 = mulTensor(z1,W2) + b2:double()
local z2 = sigmoid(a2)
local a3 = mulTensor(z2,W3) + b3:double()
local y = softmax(a3)
return y
end
local x, t = get_data()
local network = init_network()
local accuracy_cnt = 0
for i = 1, x:size()[1] do
local y = predict(network, x[i])
local value, indices = torch.max(y, 1) -- torch.max( , 1)Get the maximum value in the col direction with
local p = tensor2scalar(indices) -1 --Since it is a python output, the start of the array is 0, the array of Lua(table type)Starts from 1, so shift by 1
if p == t[i] then
accuracy_cnt = accuracy_cnt + 1
end
end
print("Accuracy:"..(accuracy_cnt/x:size()[1]))
Execution example
$ th neuralnet_mnist2.lua
Accuracy:0.9352
With this, I was able to output the same output as the original.
Recommended Posts