[PYTHON] I tried CNN fine tuning with Resnet
Fine tuning with Python Resnet
** This article introduces fine tuning using Resnet. ** **
This time, in order to verify how effective fine tuning is, it is normal. I would like to compare [Model created by CNN] and [Fine-tuned model].
Development environment
I learned with Google Colab, but I will describe it as possible even with local assumptions. python : 3.7.0 keras : 2.4.3 tensorflow : 2.2.0
What is Resnet
Resnet is a [learned] convolutional neural network with over 1 million images in a database called Imagenet. And this network is also called Resnet 50, which has 50 layers and can classify 1000 categories.
What is fine tuning?
Retraining the weights of the entire model, with the trained network weights as the initial value. Therefore, after using the above Resnet50, we aim to make a better discriminator by re-learning.
Image to use
It seemed interesting, so I tried using 3 types of Japanese cigarettes. ** Mobius: 338 sheets ** ** Seven Stars: 552 sheets ** ** Winston: 436 sheets **
This time it is a verification of fine tuning, so the above image was pulled from the Internet.
Normal CNN
The configuration is as follows.
Learning this resulted in the following results.
It's pretty bad because we haven't done any pre-processing such as cropping the image. Is it around [60%] in the test data?
Source
normal_cnn.py
from PIL import Image
import numpy as np
import glob
import os
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import MaxPooling2D
from keras.layers import Conv2D, Flatten, Dense, Dropout
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
root = "tobacco_dataset"
folder = os.listdir(root)
image_size = 224
dense_size = len(folder)
epochs = 30
batch_size = 16
X = []
Y = []
for index, name in enumerate(folder):
dir = "./" + root + "/" + name
print("dir : ", dir)
files = glob.glob(dir + "/*")
print("number : " + str(files.__len__()))
for i, file in enumerate(files):
try:
image = Image.open(file)
image = image.convert("RGB")
image = image.resize((image_size, image_size))
data = np.asarray(image)
X.append(data)
Y.append(index)
except :
print("read image error")
X = np.array(X)
Y = np.array(Y)
X = X.astype('float32')
X = X / 255.0
Y = np_utils.to_categorical(Y, dense_size)
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.15)
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(image_size, image_size, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(dense_size, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
result = model.fit(X_train, y_train, validation_split=0.15, epochs=epochs, batch_size=batch_size)
x = range(epochs)
plt.title('Model accuracy')
plt.plot(x, result.history['accuracy'], label='accuracy')
plt.plot(x, result.history['val_accuracy'], label='val_accuracy')
plt.xlabel('Epoch')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5), borderaxespad=0, ncol=2)
name = 'tobacco_dataset_reslut.jpg'
plt.savefig(name, bbox_inches='tight')
plt.close()
Fine tuning with Resnet
Since the composition is quite deep and can not be pasted on Qiita, I will describe the URL https://github.com/daichimizuno/cnn_fine_tuning/blob/master/finetuning_layer.txt
In addition to the original Resnet, the activation function Relu and dropout are added by 0.5. By the way, the following sources are the sources when using Resnet. It seems that Keras contains a library for Resnet.
ResNet50 = ResNet50(include_top=False, weights='imagenet',input_tensor=input_tensor)
Learning this resulted in the following results.
Is it around [85%] in the test data?
Source
resnet_fine_tuning.py
from PIL import Image
import numpy as np
import glob
import os
from keras.utils import np_utils
from keras.models import Sequential, Model
from keras.layers import Flatten, Dense,Input, Dropout
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
from keras.applications.resnet50 import ResNet50
from keras import optimizers
root = "tobacco_dataset"
folder = os.listdir(root)
image_size = 224
dense_size = len(folder)
epochs = 30
batch_size = 16
X = []
Y = []
for index, name in enumerate(folder):
dir = "./" + root + "/" + name
print("dir : ", dir)
files = glob.glob(dir + "/*")
print("number : " + str(files.__len__()))
for i, file in enumerate(files):
try:
image = Image.open(file)
image = image.convert("RGB")
image = image.resize((image_size, image_size))
data = np.asarray(image)
X.append(data)
Y.append(index)
except :
print("read image error")
X = np.array(X)
Y = np.array(Y)
X = X.astype('float32')
X = X / 255.0
Y = np_utils.to_categorical(Y, dense_size)
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.15)
input_tensor = Input(shape=(image_size, image_size, 3))
ResNet50 = ResNet50(include_top=False, weights='imagenet',input_tensor=input_tensor)
top_model = Sequential()
top_model.add(Flatten(input_shape=ResNet50.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(dense_size, activation='softmax'))
top_model = Model(input=ResNet50.input, output=top_model(ResNet50.output))
top_model.compile(loss='categorical_crossentropy',optimizer=optimizers.SGD(lr=1e-3, momentum=0.9),metrics=['accuracy'])
top_model.summary()
result = top_model.fit(X_train, y_train, validation_split=0.15, epochs=epochs, batch_size=batch_size)
x = range(epochs)
plt.title('Model accuracy')
plt.plot(x, result.history['accuracy'], label='accuracy')
plt.plot(x, result.history['val_accuracy'], label='val_accuracy')
plt.xlabel('Epoch')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5), borderaxespad=0, ncol=2)
name = 'resnet_tobacco_dataset_reslut.jpg'
plt.savefig(name, bbox_inches='tight')
plt.close()
Summary
Comparing normal CNN and fine tuning, you can see that fine tuning has improved sharply from around Epoch 10. Since this is re-learning using Resnet, I imagine that it will propagate in the part where the correct answer rate of Resnet increases and affect tobacco discrimination, but more detailed analysis is necessary. ..
However, in any case, the result was considerably higher than that of normal CNN in terms of tobacco judgment. I think that even better results will be obtained if you select images and perform pre-processing, so if you can, please try it!
【Github】 https://github.com/daichimizuno/cnn_fine_tuning
Please point out any mistakes or unclear points. that's all
Recommended Posts