[PYTHON] I tried face recognition of the laughter problem using Keras.

Identify the laughter problem with image recognition

I was free in Corona so I made it quickly. Most of the code is reused when the image of Keyakizaka46 is recognized. If you want to recognize your favorite images with deep learning, please come! So that you can do it even for the first time, the parts that are likely to trip are also listed! !! [I think this is all you need to do with python preferences! ] (https://prog-8.com/docs/python-env)

directory

/bakusyomondai
　/data
　　/TANAKA
　　/ota
　/face
　　/ota
　　/TANAKA
　/train
　　/ota
　　/TANAKA
　/test
　　/ota
　　/TANAKA
　
　get_image.py
　detect_face/py
　devide_test_train.py
　inflation.py
　learn2.py
　Bakunin.py

For example, in the case of mac, it is as follows.

table of contents

1, download images to learn 2, Face detection of downloaded image 3, Inflating the image to be learned 4, Learning (Deep Learninng production) 5, Discriminate images using the learned model

1, download images to learn

First, create and save the following file called get_image.py. Click here for how to download automatically from google search images

get_image.py

from icrawler.builtin import BingImageCrawler
import sys
import os

argv = sys.argv

if not os.path.isdir(argv[1]):
    os.makedirs(argv[1])


crawler = BingImageCrawler(storage = {"root_dir" : argv[1]})
crawler.crawl(keyword = argv[2], max_num = 1000)

If you execute this python file in the terminal as follows, the image will be downloaded. For example, if you want to download Tanaka from Bakusho Mondai,

$ python */get_image.py */data/TANAKA Bakusho Mondai Tanaka

The download will start when you execute.

As an image

$python */get_image.py Directory of the location you want to save Name you want to search

By the way

I think that a lot of asterisks will appear in the code this time like * / get_image.py, but this time I dare to hide it with an asterisk. Please refer to the link below when checking the file directory on your own computer. For windows For Mac

2. Face detection of downloaded images

I think the contents of the file created earlier are as follows. If there is a photo other than Mr. Ota, I deleted it, and if there are people other than Mr. Ota, I cut out the face of only Mr. Ota.

First of all, this time, I used the following code.

detect_face.py

import glob
import os
import cv2

names = ['ota','TANAKA']
out_dir = "*/bakusyomondai/face/"
os.makedirs(out_dir, exist_ok=True)
for i in range(len(names)):
   
    in_dir = "*/bakusyomondai/data/"+names[i]+"/*.jpg "
    in_jpg = glob.glob(in_dir)
    os.makedirs(out_dir + names[i], exist_ok=True)
    # print(in_jpg)
    print(len(in_jpg))
    for num in range(len(in_jpg)):
        image=cv2.imread(str(in_jpg[num]))
        if image is None:
            print("Not open:",num)
            continue

        image_gs = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        cascade = cv2.CascadeClassifier("*/haarcascade_frontalface_alt.xml")

       
        face_list=cascade.detectMultiScale(image_gs, scaleFactor=1.1, minNeighbors=2,minSize=(64,64))
       
        if len(face_list) > 0:
            for rect in face_list:
                x,y,width,height=rect
                image = image[rect[1]:rect[1]+rect[3],rect[0]:rect[0]+rect[2]]
                if image.shape[0]<64:
                    continue
                image = cv2.resize(image,(64,64))
                
                fileName=os.path.join(out_dir+"/"+names[i],str(num)+".jpg ")
                cv2.imwrite(str(fileName),image)
                print(str(num)+".I saved the jpg.")
       
        else:
            print("no face")
            continue
        print(image.shape)

When you run this code

Since a cascade file called haarcascade_frontalface_alt.xml is used for face detection, Download from here!

3. Divide the image into training data and test data.

devide_test_train.py

import shutil
import random
import glob
import os
names = ['ota','TANAKA']
os.makedirs("*/bakusyomondai/test", exist_ok=True)

for name in names:
    in_dir = "*/bakusyomondai/face/"+name+"/*"
    in_jpg=glob.glob(in_dir)
    img_file_name_list=os.listdir("*/bakusyomondai/face/"+name+"/")
    #img_file_name_Shuffle list, 20% of which test_Put in image directory
    random.shuffle(in_jpg)
    os.makedirs('*/bakusyomondai/test/' + name, exist_ok=True)
    for t in range(len(in_jpg)//5):
        shutil.move(str(in_jpg[t]), "*/bakusyomondai/test/"+name)

4. Inflating training data

As expected, the number of images is small, so I will inflate the images.

inflation.py

import os
import cv2
import glob
from scipy import ndimage

names = ['ota','TANAKA']
os.makedirs("*/bakusyomondai/train", exist_ok=True)
for name in names:
    in_dir = "*/bakusyomondai/face/"+name+"/*"
    out_dir = "*/bakusyomondai/train/"+name
    os.makedirs(out_dir, exist_ok=True)
    in_jpg=glob.glob(in_dir)
    img_file_name_list=os.listdir("*/bakusyomondai/face/"+name+"/")
    for i in range(len(in_jpg)):
        #print(str(in_jpg[i]))
        img = cv2.imread(str(in_jpg[i]))
        
        for ang in [-10,0,10]:
            img_rot = ndimage.rotate(img,ang)
            img_rot = cv2.resize(img_rot,(64,64))
            fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+".jpg ")
            cv2.imwrite(str(fileName),img_rot)
           
            img_thr = cv2.threshold(img_rot, 100, 255, cv2.THRESH_TOZERO)[1]
            fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+"thr.jpg ")
            cv2.imwrite(str(fileName),img_thr)
            
            img_filter = cv2.GaussianBlur(img_rot, (5, 5), 0)
            fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+"filter.jpg ")
            cv2.imwrite(str(fileName),img_filter)

When executed, the image will be inflated as shown below.

5, start of learning

learn2.py

import keras
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.layers.core import Dense, Dropout, Activation, Flatten
import numpy as np
from sklearn.model_selection import train_test_split
from PIL import Image
import glob

folder = ['ota','TANAKA']
image_size = 50

X_train = []
y_train = []
for index, name in enumerate(folder):
    dir = "*/bakusyomondai/train/" + name
    files = glob.glob(dir + "/*.jpg ")
    for i, file in enumerate(files):
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        data = np.asarray(image)
        X_train.append(data)
        y_train.append(index)

X_train = np.array(X_train)
y_train = np.array(y_train)

folder = ['ota','TANAKA']
image_size = 50

X_test = []
y_test = []
for index, name in enumerate(folder):
    dir = "*/bakusyomondai/test/" + name
    files = glob.glob(dir + "/*.jpg ")
    for i, file in enumerate(files):
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        data = np.asarray(image)
        X_test.append(data)
        y_test.append(index)

X_test = np.array(X_test)
y_test = np.array(y_test)

X_train = X_train.astype('float32')
X_train = X_train / 255.0

X_test = X_test.astype('float32')
X_test = X_test / 255.0

#Convert the correct label format
y_train = np_utils.to_categorical(y_train, 2)
#Convert the correct label format
y_test = np_utils.to_categorical(y_test, 2)

#Build CNN
model = Sequential()

model.add(Conv2D(32, (3, 3), padding='same',input_shape=X_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(2))
model.add(Activation('softmax'))

#compile
model.compile(loss='categorical_crossentropy',optimizer='SGD',metrics=['accuracy'])

history = model.fit(X_train, y_train, epochs=100)
print(model.evaluate(X_test, y_test))
model.save("*/bakusyomonadai/bakusyo_model.h5")

y_train = np_utils.to_categorical(y_train, 4)
#Convert the correct label format
y_test = np_utils.to_categorical(y_test, 4)

model.add(Dense(4))

If you change this part, it will work! !!

6. Discriminate images using the learned model

kakunin.py

from keras.models import load_model
import numpy as np
from keras.preprocessing.image import img_to_array, load_img

jpg_name = '*/hirate/000002.jpg'
my_model='*/bakusyomondai/bakusyo_model.h5'

model=load_model(my_model)

img_path = (jpg_name)
img = img_to_array(load_img(img_path, target_size=(50,50)))
img_nad = img_to_array(img)/255
img_nad = img_nad[None, ...]

label=['ota','TANAKA']
pred = model.predict(img_nad, batch_size=1, verbose=0)
score = np.max(pred)
pred_label = label[np.argmax(pred[0])]
print('name:',pred_label)
print('score:',score)

This time I tried to see how to distinguish when Hirate of Keyakizaka in a different file is put.

I recognized that it was Mr. Ota with a 99.9% probability lol

By the way, If you judge by the real Mr. Tanaka It will determine it properly! !! !!

This time, I tried to summarize the flow of image recognition using deep learning. I could not omit it as much as possible so that even beginners can do it. (I had a hard time, but lol) I think you can do it with your favorite actors and characters, so please come and join us! !!

Quote https://qiita.com/nirs_kd56/items/bc78bf2c3164a6da1ded