MPSYOKOHAMA 10th python-mnist, a library of handwritten digit character training data and teacher data used in the DNN implementation, runs in the windows environment. I didn't have it, so a memo when I forced it to move
-Install python-mnist 0.3 with pip ・ From MINST DATABASE site, train-images-idx3-ubyte.gz: train-labels-idx1-ubyte.gz: download -Create a folder named mnist in the directory where python is running. -Unzip the above two files and save them. In other words Execution environment> mninst> train-images-idx3-ubyte> train-images.idx3-ubyte (file) Execution environment> mninst> train-labels-idx1-ubyte> train-labels.idx1-ubyte (file) I made the situation. So, when I try to read it, I first specify the mnist folder under the execution environment, but "" This is \ (half-width) for windows.
from mnist import MNIST
mndata = MNIST('.\mnist')
And then to read the file
train_img, train_label = mndata.load_training()
When I executed it, it was rejected due to a permission problem. .. .. I tried playing with the permissions of various files, but I couldn't solve it.
PermissionError: [Errno 13] Permission denied: '.\\mnist\\train-labels-idx1-ubyte'
However, if you directly insert import os and read it, you can access the file. Hmmm.
with open('.\mnist\\train-labels-idx1-ubyte\\train-labels.idx1-ubyte', 'rb') as f:
line = f.readline()
print(line)
>>>b'\x00\x00\x08\x01\x00\x00\xea....
It can't be helped, so let's load it directly while referring to the library.
C:\Python34\Lib\site-packages\mnist
With reference to loader.py here, I decided to write a code that can only do what I need this time by imitating it.
I really wanted to use something like os.join to make it work in any storage location, but I gave up because of lack of ability. I put a file directly under mnist and read it. Execution environment> mninst> train-images.idx3-ubyte (file) Execution environment> mninst> train-labels.idx1-ubyte (file)
WMINST.py
# coding: utf-8
import os
import struct
from array import array
class MNIST(object):
def __init__(self):
self.train_img_fname = 'train-images-idx3-ubyte'
self.train_lbl_fname = 'train-labels-idx1-ubyte'
self.train_images = []
self.train_labels = []
def load_training(self): #Create a folder called mninst in the working folder and train in it-images.idx3-ubyte,train-labels.idx1-I put ubyte.
ims, labels = self.load(('.\mnist\\train-images.idx3-ubyte'),
('.\mnist\\train-labels.idx1-ubyte'))
self.train_images = ims
self.train_labels = labels
return ims, labels
@classmethod
def load(cls, path_img, path_lbl):
with open(path_lbl, 'rb') as file:
magic, size = struct.unpack(">II", file.read(8))
if magic != 2049:
raise ValueError('Magic number mismatch, expected 2049,'
'got {}'.format(magic))
labels = array("B", file.read())
with open(path_img, 'rb') as file:
magic, size, rows, cols = struct.unpack(">IIII", file.read(16))
if magic != 2051:
raise ValueError('Magic number mismatch, expected 2051,'
'got {}'.format(magic))
image_data = array("B", file.read())
images = []
for i in range(size):
images.append([0] * rows * cols)
for i in range(size):
images[i][:] = image_data[i * rows * cols:(i + 1) * rows * cols]
return images, labels
I was addicted to specifying the file path like `'. \ Mnist \\ train-labels.idx1-ubyte'```, and after the first mnist folder \\ `` Connect with two. (``` \ is \ (half-width))
I gave this file an appropriate name (WMINST.py), saved it in the working directory, and read it, and it worked. The reason why it becomes MNIST () and not (. \ Mnist) is because I wrote it so that the file path can be loaded directly.
from WMNIST import MNIST
mndata = MNIST()
train_img, train_label = mndata.load_training()
Apparently it worked.
Recommended Posts