[PYTHON] Sort Fashion-MNIST data and save as PNG file

Introduction

Do you do machine learning? ＞ Greeting

I used to use Chainer, but for some reason I decided to use PyTorch or Keras. So, the machine learning tutorial is the MNIST dataset of handwritten numbers, but the tutorial of TensorFlow uses the data of Fashion-MNIST with a slight twist. I am.

This tutorial is pretty straightforward, but for now it's helpful to take a quick look at what data you're training. So let's download and sort the data used for Fashion-MNIST, the clothing classification problem.

I'll put the code below.

https://github.com/kaityo256/fashion_mnist_dump

How to use

Just run it. I think you need TensorFlow Version 2 or later.

python fashion_mnist_dump.py 

When executed, the following directories will be dug and the data with the corresponding labels will be saved in serial numbers.

test
├ Bag
├ Boot
├ Coat
├ Dress
├ Pullover
├ Sandal
├ Shirt
├ Sneaker
├ Top
└ Trouser

train
├ Bag
├ Boot
├ Coat
├ Dress
├ Pullover
├ Sandal
├ Shirt
├ Sneaker
├ Top
└ Trouser

There are 10 types of labels. The test data is 1000 sheets for each label, totaling 10,000 sheets, and the training data is 6000 sheets, totaling 60,000 sheets, for a total of 70,000 sheets.

For example, the test data Sandal

If Dress

It is sorted and saved as.

script

The script is also short, so I will post it all.

import os

import IPython
from PIL import Image
import numpy as np
import tensorflow as tf
from tensorflow import keras

fashion_mnist = keras.datasets.fashion_mnist
train, test = fashion_mnist.load_data()

dirnames = ['Top', 'Trouser', 'Pullover', 'Dress',
            'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Boot']


def save_img(filename, data):
    img = Image.new("L", (28, 28))
    pix = img.load()
    for i in range(28):
        for j in range(28):
            pix[i, j] = int(data[j][i])
    img2 = img.resize((28*5, 28*5))
    img2.save(filename)


def dump(data, dhead):
    for i in range(10):
        dname = "{}/{}".format(dhead, dirnames[i])
        if os.path.isdir(dname) is False:
            os.makedirs(dname)
    images, labels = data
    count = [0]*10
    for i in range(len(images)):
        index = labels[i]
        filename = "{}/{}/{}.png ".format(dhead, dirnames[index], count[index])
        save_img(filename, images[i])
        count[index] += 1
        print(filename)


dump(test, "test")
dump(train, "train")

I don't think it's particularly difficult, but I'll explain it briefly.

First, download the data.

fashion_mnist = keras.datasets.fashion_mnist
train, test = fashion_mnist.load_data()

fashion_mnist.load_data () returns the training and test datasets.

Each dataset consists of image data and label data. Each is a NumPy array.

So

train_images, train_labels = train

You can get the image and label by doing something like that.

The image is a NumPy array with test data of 10000 x 28 x 28 and training data of 60000 x 28 x 28. The type is ʻuint8`, that is, an integer from 0 to 255, so you can save it as an image by using the grayscale brightness as it is. The following function does that.

def save_img(filename, data):
    img = Image.new("L", (28, 28))
    pix = img.load()
    for i in range(28):
        for j in range(28):
            pix[i, j] = int(data[j][i])
    img2 = img.resize((28*5, 28*5))
    img2.save(filename)

Create a 28 x 28 pixel grayscale image with ʻImage.new ("L", (28, 28)) and get the pixel data with ʻimg.load (). All you have to do now is poke a NumPy array into that data (you'll have to cast it to ʻint`).

Since it is a little small as it is, it is enlarged 5 times and saved with ʻimg.resize ((28 * 5, 28 * 5))`.

The dump (data, dhead) function is called by putting the file name and data in this save_img.

data is test or training data, and dhead is test or training, and they are distinguished by the directory name.

It's not particularly difficult, but when using it for learning, it is more convenient to convert this data to real number data from -1.0 to 1.0, and flatten it to 784 1D data instead of 28x28 2D data. Good (CNN might be better to stay in 2D).

Summary

I used TensorFlow / Keras to get Fashion-MNIST data, sorted it, and saved it as a file. It's interesting to take a quick look at the data. For example

I thought, "Oh, it's Gap."

There are things that humans do not understand even if they see it [^ 1]

Something like pants

After all, there are things I don't understand ...

[^ 1]: The correct answer (label) is a bag.

Anyway, machine learning libraries are developing fast and it's hard to catch up ...