[PYTHON] [MNIST] I tried Fine Tuning using the ImageNet model.
① Introduction
When studying AI, I would like to study ** Fine Tuning ** as well. This time, I would like to implement ** MNIST ** using ** ImageNet trained DenseNet121 model **. The library uses ** keras **.
② What is fine tuning?
** Fine Tuning ** means ** fine-tuning ** when translated into Japanese. ** Fine-tuning ** in AI is a method of reusing a part of an existing model to build a new model.
■ Difference between fine tuning and transfer learning
Similar to ** Fine Tuning ** is ** Transfer Learning **. ** Fine tuning ** and ** Transfer learning ** are always messed up, so I'll organize them here.
-** Fine tuning **
** Fine tuning ** trains again with the weight of the trained model as the initial value. The merit is that the weight of the existing model can be additionally learned as the initial value, so that the model can be constructed more efficiently and with high accuracy.
-** Transfer learning **
** Transfer learning ** fixes the weights of the trained model and trains using only the added layers. The merit is that additional training is performed based on the trained model, so the model can be built with less data and in a short time.
■ So which one is this time?
I would like to reuse the image classification model that has already been completed to classify numbers. This time, I would like to use the "model of image classification with weights learned by ImageNet" to "classify numerical images". So, this time it will be ** fine tuning **.
③ Fine tuning method
I would like to use DenseNet121 for the model that is the ** fine tuning source **. This model is a model that is lightweight but has a good recognition rate.
** MNIST training data ** is ** 28 x 28 x 1ch **, ** DenseNet121 input layer ** is ** 32 or more x 32 or more x 3ch ** , These need to be combined. There are two ways to match.
** ・ How to match MNIST images ** → How to change the learning image to 32 or more x 32 or more x 3ch according to the input layer of the model
**-How to change the input layer of the model ** → How to change the input layer of the model to 1ch according to the learning image
■ How to match MNIST images
The input format of DenseNet121 is (height = 32 or more, width = 32 or more, channels = 3ch).
The image format of MNIST is (height = 28, width = 28, channels = 1ch). The training image size is 32 or more x 32 or more x3ch is required from the network definition of DenseNet121, so perform the following processing.
--Resize image to 64x64 --Convert from grayscale 1ch to RGB3ch
■ How to change the input layer of the model
The input format of DenseNet121 is (height = 32 or more, width = 32 or more, channels = 3ch).
The image format of MNIST is (height = 28, width = 28, channels = 1ch). The training image size is 32 or more x 32 or more x3ch is required from the network definition of DenseNet121, so perform the following processing.
--Resize image to 64x64 --Added 1ch input layer
④ How to match MNIST images
Basically, fine tuning can be performed by following the steps ➊ to ➎ below.
➊ Download learned weights
With the API of " keras.applications.densenet.DenseNet121 ", you can easily ** download the weights trained by imagenet with Densenet121 **. It's hard to download every time (33MB), so save it in Google Drive.
from keras.applications.densenet import *
model = DenseNet121(weights='imagenet')
model.save(fullpath)
➋ MNIST data conversion
Converts MNIST images from (28, 28, 1) to (64, 64, 3) according to the input format (height, width, channels) of DenseNet121.
import cv2
import numpy as np
from keras.datasets import mnist
img_w, img_h, img_ch = 64, 64, 3
num_classes = 10
def pretreatment(x):
#Resize image
X = []
for i in range(len(x)):
dst = cv2.resize(x[i], (img_h, img_w), interpolation=cv2.INTER_CUBIC)
dst = cv2.cvtColor(dst, cv2.COLOR_GRAY2RGB)
X.append(dst)
#Reshape
X = np.array(X).reshape(len(x), img_h, img_w, img_ch)
#Change data in the range 0 to 255 to the range 0 to 1
ret = X.astype('float32') / 255
return ret
#■ Read MNIST data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_valid, y_train, y_valid = train_test_split(x_train, y_train, test_size=0.2)
#■ Learning(Train)data
x_train = pretreatment(x_train)
y_train = np_utils.to_categorical(y_train, num_classes)
#■ Learning(Valid)data
x_valid = pretreatment(x_valid)
y_valid = np_utils.to_categorical(y_valid, num_classes)
#■ Test data
x_test = pretreatment(x_test)
y_test = np_utils.to_categorical(y_test, num_classes)
➌ Build a network of DenseNet121
(1) DneseNet121 network construction
With keras, you can easily ** build a network of Densenet121 ** by using the API of " keras.applications.densenet.DenseNet121 ". By setting include_top to False, you can prevent the fully connected layer on the output layer side of the network from being included. Also, do not read weights at this point.
input_tensor = x_train.shape[1:]
model = DenseNet121(include_top=False, weights=None, input_shape=input_tensor)
(2) Addition of output layer
Build a fully connected layer on the output layer side of the network DenseNet121 (ImageNet) discriminates 1000 classes as image classification, but since this time we will implement MNIST, we will make it possible to classify into 10 classifications from 0 to 9.
from keras.layers import *
from keras.models import Model
x = Flatten()(model.layers[-1].output)
x = Dense(10, activation="softmax")(x)
Model(model.inputs, x)
➍ Read learned weights
After configuring the network, load the ** weights downloaded in ➊ **. The weights are loaded by this " load_weights "method.
model.load_weights(fullpath, by_name=True)
➎ Learn
After that, you can do ** Fine Tuning ** by learning as usual.
➏ Execution result
● TEST: Verification result
Test loss : 0.01561666838824749
Test accuracy: 0.9944000244140625
⑤ How to change the input layer of the model
Basically, fine tuning can be performed by following the steps ➊ to ➍ below.
➊ Download learned weights
With the API of " keras.applications.densenet.DenseNet121 ", you can easily ** download the weights trained by imagenet with Densenet121 **. It's hard to download every time (33MB), so save it in Google Drive.
from keras.applications.densenet import *
model = DenseNet121(weights='imagenet')
model.save(fullpath)
➋ MNIST data conversion
Converts MNIST images from (28, 28, 1) to (64, 64, 1) according to the input format (height, width, channels) of DenseNet121.
import cv2
import numpy as np
from keras.datasets import mnist
img_w, img_h, img_ch = 64, 64, 1
num_classes = 10
def pretreatment(x):
#Resize image
X = []
for i in range(len(x)):
dst = cv2.resize(x[i], (img_h, img_w), interpolation=cv2.INTER_CUBIC)
# dst = cv2.cvtColor(dst, cv2.COLOR_GRAY2RGB)← Do not execute because it is 1ch
X.append(dst)
#Reshape
X = np.array(X).reshape(len(x), img_h, img_w, img_ch)
#Change data in the range 0 to 255 to the range 0 to 1
ret = X.astype('float32') / 255
return ret
#■ Read MNIST data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_valid, y_train, y_valid = train_test_split(x_train, y_train, test_size=0.2)
#■ Learning(Train)data
x_train = pretreatment(x_train)
y_train = np_utils.to_categorical(y_train, num_classes)
#■ Learning(Valid)data
x_valid = pretreatment(x_valid)
y_valid = np_utils.to_categorical(y_valid, num_classes)
#■ Test data
x_test = pretreatment(x_test)
y_test = np_utils.to_categorical(y_test, num_classes)
➌ Build a new network using DenseNet121
(1) Add your own input layer before the input layer of DenseNet121
from keras.layers import *
from keras.models import Model, Sequential
input_tensor = x_train.shape[1:]
input_model = Sequential()
input_model.add(InputLayer(input_shape=input_tensor))
input_model.add(Conv2D(3, (3, 3), padding='same'))
input_model.add(BatchNormalization())
input_model.add(Activation('relu'))
(2) DneseNet121 network construction
With keras, you can easily ** build a network of Densenet121 ** by using the API of " keras.applications.densenet.DenseNet121 ". By setting include_top to False, you can prevent the fully connected layer on the output layer side of the network from being included. Also, do not read weights at this point.
model = DenseNet121(include_top=False, weights=None, input_tensor=input_model.output)
(3) Addition of output layer
DenseNet121 (ImageNet) discriminates 1000 classes as image classification, but since this time we will implement MNIST, we will make it possible to classify into 10 classifications from 0 to 9.
x = Flatten()(model.layers[-1].output)
x = Dense(num_classes, activation="softmax")(x)
return Model(model.inputs, x)
➍ Read learned weights
After configuring the network, load the ** weights downloaded in ➊ **. The weights are loaded by this " load_weights "method.
model.load_weights(fullpath, by_name=True)
➎ Learn
After that, if you learn as usual, you can do ** Fine Tuning **.
➏ Execution result
● TEST: Verification result
Test loss : 0.019007695838809013
Test accuracy: 0.9934999942779541
⑥ Source code
When training this time, ImageDataGenerator is used to train including the following rotated / reduced images, and generalization performance is also taken into consideration.
** Click here for the entire source code ** ▶ github
⑦ and above
Thank you for your hard work. Since it is based on the ** ImageNet trained DenseNet 121 model **, the amount of calculation for 1 epoc is large, but since it is ** Fine Tuning **, it almost converges in about 30 epoc. The correct answer rate and loss value are also reasonably good. This time, we read the weights for ** FineTuning **, but it can be applied to cases such as when learning is performed halfway and learning is started again, so it is good to understand this procedure.
Recommended Posts