[PYTHON] Are Alice Hirose and Martin (Lotte) really similar?
Obon holidays are also a wonderful place.
Suddenly, the faces of actress Alice Hirose and Martin of Chiba Lotte Marines seem to be very similar, and Alice Hirose himself made an official comment, which seems to be a hot topic in the streets these days. (News)
So, this time, I tried to verify by image recognition whether Alice Hirose and Martin are really similar.
Learning environment
Window 10 python 3.7.6 jupyter lab
The page that I used as a reference
The contents of this implementation are based on the following page (rather, almost copy and paste). Thank you very much.
Reference page: I tried face recognition for the laughter problem using Keras.
This time, I am working directly under the C drive, and the directory path is defined as an absolute path.
Implementation content
Image collection part
First, collect images for learning. Here, I use icrawler to collect quickly. This time, I don't use command line arguments, but embed the pathname and search word in the python code. The number of collections is set to 300, but it has not been collected so much.
#-*- coding:utf-8 -*-
from icrawler.builtin import BingImageCrawler
import sys
import os
#Image collection of Martin
filepath = "C:\\MartinVSAris\\data\\Martin"
search = "Martin Lotte"
#Image collection of Alice Hirose
#filepath = "C:\\MartinVSAris\\data\\Aris"
#search = "Alice Hirose"
if not os.path.isdir(filepath):
os.makedirs(filepath)
crawler = BingImageCrawler(storage = {"root_dir" : filepath})
crawler.crawl(keyword = search, max_num = 300)
The collected results are like this.
From here, there were images of different people and images that were not people, so I deleted them. However, since I am doing it visually, there is a possibility that Alice Hirose has a younger sister, Suzu Hirose. (Maybe I should have made that a problem lol)
Cut out the face part from the image
Cut out the face part from the images of Alice Hirose and Martin collected earlier and save it. Unlike the reference page, this time we are using dlib for face detection.
import glob
import os
import dlib
import cv2
def face_detect(img_list,save_dir):
#Face detection classifier
detector = dlib.get_frontal_face_detector()
#Put the candidate image on the classifier
for (i,img_path) in enumerate(img_list):
#
print(img_path) #Image path name display
index = i + 1 #For the file name of the image to save
#Image loading
img = cv2.imread(img_path, cv2.IMREAD_COLOR)
#Get coordinate information of face area
cv_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
faces = detector(cv_img, 1)
#Face image generation
img_count = 1
for face in faces:
#Get candidate image size
height, width = img.shape[:2]
#Get the coordinate points of the face area
top = face.top()
bottom = face.bottom()
left = face.left()
right = face.right()
#Ignore if face is not recognized
if not top < 0 and left < 0 and bottom > height and right > width:
print("No facial recognition.")
img_count += 1
break
else:
#Face area trimming
dst_img = img[top:bottom, left:right]
#Face image size Normalized and saved
face_img = cv2.resize(dst_img, (64,64))
new_img_name = save_dir + '//' + str(index) + '.jpg'
cv2.imwrite(new_img_name, face_img)
img_count += 1
del top,bottom,left,right
names = ['Martin','Aris']
out_dir = "C:\\MartinVSAris\\faces\\"
os.makedirs(out_dir, exist_ok = True)
for i in range(len(names)):
in_dir = "C:\\MartinVSAris\\data\\" + names[i] + "\\*.jpg "
save_dir = out_dir + names[i]
in_jpg = glob.glob(in_dir)
os.makedirs(out_dir + names[i], exist_ok = True)
# print(in_jpg)
print(len(in_jpg)) #Display the number of stored images
face_detect(in_jpg,save_dir) #Function execution part of face detection
As a result of turning the above code, it looks like this.
Alice Hirose
Martin player
I can't say if they are similar or not ...
Divide the image into training data and test data
This time, the part that separates the training data and the test data on the reference page and the part that inflates the training data are combined into one code. In addition, since the collected image data was small, the rotation angle of the image is increased compared to the reference page.
import shutil
import random
import glob
import os
import cv2
import glob
from scipy import ndimage
names = ['Martin','Aris']
#Create folder
os.makedirs("C:\\MartinVSAris\\test", exist_ok=True)
os.makedirs("C:\\MartinVSAris\\train", exist_ok=True)
for name in names:
in_dir = "C:\\MartinVSAris\\faces\\" + name + "\\*"
in_jpg=glob.glob(in_dir)
img_file_name_list=os.listdir("C:\\MartinVSAris\\\faces\\" + name + "\\")
#img_file_name_Shuffle list, 20% of which test_Put in image directory
random.shuffle(in_jpg)
os.makedirs("C:\\MartinVSAris\\test\\" + name, exist_ok=True)
for t in range(len(in_jpg)//5):
shutil.move(str(in_jpg[t]), "C:\\MartinVSAris\\test\\" + name)
in_jpg=glob.glob(in_dir)
img_file_name_list=os.listdir("C:\\MartinVSAris\\faces\\" + name + "\\")
os.makedirs("C:\\MartinVSAris\\train\\" + name, exist_ok=True)
out_dir = "C:\\MartinVSAris\\train\\" + name
for i in range(len(in_jpg)):
img = cv2.imread(str(in_jpg[i]))
for ang in [-30,-20,-10,0,10,20,30]:
img_rot = ndimage.rotate(img,ang)
img_rot = cv2.resize(img_rot,(64,64))
fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+".jpg ")
cv2.imwrite(str(fileName),img_rot)
#Threshold change
img_thr = cv2.threshold(img_rot, 100, 255, cv2.THRESH_TOZERO)[1]
fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+"thr.jpg ")
cv2.imwrite(str(fileName),img_thr)
#Blur
img_filter = cv2.GaussianBlur(img_rot, (5, 5), 0)
fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+"filter.jpg ")
cv2.imwrite(str(fileName),img_filter)
Learning
All you have to do is learn.
import keras
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.layers.core import Dense, Dropout, Activation, Flatten
import numpy as np
from sklearn.model_selection import train_test_split
from PIL import Image
import glob
names = ['Martin','Aris']
image_size = 50
X_train = []
y_train = []
for index, name in enumerate(folder):
dir = "C:\\MartinVSAris\\train\\" + name
files = glob.glob(dir + "\\*.jpg ")
for i, file in enumerate(files):
image = Image.open(file)
image = image.convert("RGB")
image = image.resize((image_size, image_size))
data = np.asarray(image)
X_train.append(data)
y_train.append(index)
X_train = np.array(X_train)
y_train = np.array(y_train)
folder = ['Martin','Aris']
image_size = 50
X_test = []
y_test = []
for index, name in enumerate(folder):
dir = "C:\\MartinVSAris\\test\\" + name
files = glob.glob(dir + "\\*.jpg ")
for i, file in enumerate(files):
image = Image.open(file)
image = image.convert("RGB")
image = image.resize((image_size, image_size))
data = np.asarray(image)
X_test.append(data)
y_test.append(index)
X_test = np.array(X_test)
y_test = np.array(y_test)
X_train = X_train.astype('float32')
X_train = X_train / 255.0
X_test = X_test.astype('float32')
X_test = X_test / 255.0
#Convert the correct label format
y_train = np_utils.to_categorical(y_train, 2)
#Convert the correct label format
y_test = np_utils.to_categorical(y_test, 2)
#Build CNN
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',input_shape=X_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(2))
model.add(Activation('softmax'))
#compile
model.compile(loss='categorical_crossentropy',optimizer='SGD',metrics=['accuracy'])
epochs = 100
history = model.fit(X_train, y_train, epochs=epochs)
print(model.evaluate(X_test, y_test))
model.save("C:\\MartinVSAris\\learn_model.h5")
For the time being, let's check the accuracy rate and loss function by graphing them.
import matplotlib.pyplot as plt
%matplotlib inline
x = range(epochs)
fig1 = plt.figure()
plt.plot(x, history.history['accuracy'])
plt.title("accuracy")
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.show()
fig1.savefig("accuracy.png ")
fig2 = plt.figure()
plt.plot(x, history.history['loss'])
plt.title("loss")
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.show()
fig2.savefig("loss.png ")
The following two figures show the correct answer rate and the loss function of the learning result this time, but since the correct answer rate is sufficiently close to 1 and the loss function is also close to 0, it can be said that learning is possible without problems. ..
Discrimination result
Let's make a decision. The code is here.
from keras.models import load_model
import numpy as np
from keras.preprocessing.image import img_to_array, load_img
jpg_name = 'C:\\hanbetsu.jpg'
my_model='C:\\MartinVSAris\\learn_model.h5'
model=load_model(my_model)
img_path = (jpg_name)
img = img_to_array(load_img(img_path, target_size=(50,50)))
img_nad = img_to_array(img)/255
img_nad = img_nad[None, ...]
label = ['Martin','Aris']
pred = model.predict(img_nad, batch_size=1, verbose=0)
score = np.max(pred)
pred_label = label[np.argmax(pred[0])]
print('name:',pred_label)
print('score:',score)
Let's use the above code to determine the data that is not used for training. (Sorry for the different image sizes) First of all, Alice Hirose
name: Aris
score: 0.9999968
Next, Martin
name: Martin
score: 1.0
That's right, lol. I can distinguish it firmly.
Also, I tried to identify Martin's face when Alice Hirose made an official comment,
name: Martin
score: 0.9427792
I have identified it as Martin, but since the score is lower than before, I think it can be said that there was also an element of Alice Hirose.
By the way, I think there is a story about how it is in terms of image recognition, but if you try to distinguish the image comparing the faces of two people as it is,
name: Martin
score: 1.0
Martin was the dominant player lol
Lastly, I tried to recognize the image of my sister Suzu Hirose.
name: Martin
score: 0.99986243
...(Lol)
Summary
This time, we performed face recognition of Alice Hirose and Martin by image recognition using Keras. Although there are various problems such as the small number of collected images, we were able to create a model that can distinguish between the two to some extent. It's nice to be able to study while having fun like this.
Recommended Posts