[PYTHON] [Explanation for beginners] OpenCV face detection mechanism and practice (detect MultiScale)

Overview and environment

I investigated and tried the face detection method of openCV. I will explain for beginners. The following are openCV related articles.

-Install OpenCV 3.3 and Python 3.6 on Windows 10 with Anaconda -Specify parameters in openCV face detection to quickly improve detection accuracy -Face detection from multiple image files with openCV and cut out and saved -Tips for efficiently detecting faces with large numbers of images with openCV

The verification environment is as follows.

What you could do with openCV face detection

The face was detected for the person image and surrounded by a square frame.

Multiple people are okay. Wow!

Detects even funny faces! (The person behind can also be detected)

You can also switch from face detection to nose detection.

OpenCV face detection mechanism

I haven't looked at the papers or specific algorithms to find out and confirm them, so the following explanations may be incorrect (I'd appreciate it if you could point them out). This is what the author understood in about half a day. Perhaps watching the video "Viola Jones face detection and tracking explained" will give you a better understanding, but realize the long 1.5 hours. Broke ...

Face detection method overview

Judgment A part of the entire image (read image) is cut out and judged according to various criteria. If it is judged as "not a face" even once, the subsequent judgment is not performed and the judgment device can be applied to a part of the next image. By not making all judgments, the speed of face detection processing is realized (the accuracy of individual judgments is low, but the accuracy is maintained as a whole by stacking many).

I changed the parameters of the API of openCV so that the cutting process on the left side is easy to understand, and dared to make it falsely detected. Faces are detected in a large number of frames, that is, at least this many frames are used for judgment. You can see that the entire judgment image is cut out and individually applied to the judgment device.

Judgment method

Harr-like detector overview

Face judgment is performed using a Haar-like detector. The judgment process looks like this. The central part of the nose is bright because it is exposed to light, and the area around the nose is shadowed and darkened, so the above judgment method is effective. Judgment alone is simple and quick, but on the other hand, the judgment accuracy is low. Therefore, making many judgments will improve the accuracy as a whole. There are three main types of Haar-like detectors used for the above judgments.

Weakness of judgment method

As a weak point, the detection accuracy drops significantly unless the face is in front. For example, the profile shown below could not be detected.

Also, even if I rotated my face, it could not be detected.

Python code

I verified it using the following code. It also leaves a mark of trial and error. There are many parts that I just copied from the link posted at the end. The code that can be specified when executing the trained model is written in the article "Specify parameters with face detection of openCV to quickly improve detection accuracy". I will. In addition, the code to read multiple images in the folder and cut out and save the detected face part is the article "Face detection from multiple image files with openCV and cut out and save" See / items / 457737530264572f5a5b).

import cv2

#Classifier directory(Obtained from)
# https://github.com/opencv/opencv/blob/master/data/haarcascades/
# https://github.com/opencv/opencv_contrib/blob/master/modules/face/data/cascades/

cascade_path = "./models/haarcascade_frontalface_default.xml"

#Other model files(reference)
#cascade_path = "./models/haarcascade_frontalface_alt.xml"
#cascade_path = "./models/haarcascade_frontalface_alt2.xml"
#cascade_path = "./models/haarcascade_frontalface_alt_tree.xml"
#cascade_path = "./models/haarcascade_profileface.xml"
#cascade_path = "./models/haarcascade_mcs_nose.xml"

#Files used and I / O directories
image_file = "test.jpg "
image_path = "./inputs/" + image_file
output_path = "./outputs/" + image_file

#For directory confirmation(For when things go wrong)
#import os
#print(os.path.exists(image_path))

#File reading
image = cv2.imread(image_path)

#Grayscale conversion
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

#Acquire the features of the cascade classifier
cascade = cv2.CascadeClassifier(cascade_path)

#Execution of object recognition (face recognition)
#image – CV_8U type matrix. Objects are detected in the images stored here
#objects – A vector whose elements are rectangles. Each rectangle contains the detected object
#scaleFactor – Represents the amount of reduction at each image scale
#minNeighbors – Candidate rectangles must contain at least this number of neighbor rectangles
#flags – This parameter is not used in the new cascade. For older cascades, it has the same meaning as for the cvHaarDetectObjects function.
#minSize – The minimum size an object can take. Objects smaller than this are ignored
facerect = cascade.detectMultiScale(image_gray, scaleFactor=1.1, minNeighbors=2, minSize=(30, 30))

#print(facerect)
color = (255, 255, 255) #White

#When detected
if len(facerect) > 0:
    
    #Create a rectangle that surrounds the detected face
    for rect in facerect:
        cv2.rectangle(image, tuple(rect[0:2]),tuple(rect[0:2]+rect[2:4]), color, thickness=2)

    #Saving recognition results
    cv2.imwrite(output_path, image)

Get the file of the trained model used

Face detection and Nose detection from GitHub I am using the trained model file of modules / face / data / cascades / haarcascade_mcs_nose.xml).

I haven't verified which trained model to use with stackoverflow The article "Types and effects of Cascade Classifiers that can be used with OpenCV" is helpful.

Face detection method "cascade.detectMultiScale"

The method "cascade.detectMultiScale" is used for face detection. As you can see from the Official Site, similar methods "cascade.detectMultiScale2" and " There is cascade.detectMultiScale3 ", but it seems that the basic processing does not change, only the parameters are different. I heard it on stackoverflow. ** Parameters "scaleFactor" and "minNeighbors" are very important for improving detection accuracy. ** The author's trial and error is written down in the article "Tips for efficiently detecting faces in large numbers of images with openCV". I understood it with reference to the following information.

-Try object detection (detectMultiScale) with different parameters (scaleFactor edition) -[Try object detection (detectMultiScale) with different parameters (minNeighbors)] (http://workpiles.com/2015/04/opencv-detectmultiscale-minneighbors/)

• OpenCV detectMultiScale() minNeighbors parameter

Reference link

This time, I will post the site I used for studying as a link.