I tried face recognition from the video (OpenCV: python version)
Trigger
How to save only a part of a long video using OpenCV How to take a captured image from a video (OpenCV) I used OpenCV from the candidacy video to prepare the environment that seems to be necessary for image processing. From here, I will try to process using functions and libraries. The first is face recognition.
Preparation
Since it is developed on Mac, install it from Homebrew.
brew install opencv
Therefore, under / usr / local / Cellar / opencv / 4.1.1_2 / share / opencv4 / haarcascades /, there is a part called a cascade (distributor) that is necessary for recognizing the face and the whole body. The types are described in [Face recognition using OpenCV (Haar-like feature classifier)](https://qiita.com/hitomatagi/items/04b1b26c1bc2e8081427#haar-like feature classifier).
here,
--haarcascade_frontalface_alt_tree.xml front face --haarcascade_eye.xml Both eyes --haarcascade_righteye_2splits.xml right eye --haarcascade_lefteye_2splits.xml left eye
I will try using.
Process flow
here
- The face is in the frame
- Cut out an image that will be the ROI (Region of Interest) from 1. Divided into right area and left area respectively (Baron Ashura method).
- Determine if both the right and left eyes are visible in the image in 2.
- If the conditions of 3 are met, enclose the face, right eye and left eye with a rectangle.
Is being processed.
development of
Developed based on the above processing flow.
Version judged by dividing the right eye and the left eye
import cv2
import os
import time
if __name__ == '__main__':
cap = cv2.VideoCapture('one_minutes.mp4')
cap_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
cap_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = cap.get(cv2.CAP_PROP_FPS)
fourcc = cv2.VideoWriter_fourcc('m','p','4','v')
writer = cv2.VideoWriter('detect_face.mp4',fourcc, fps, (cap_width, cap_height))
cascade_base_path = "/usr/local/Cellar/opencv/4.1.1_2/share/opencv4/haarcascades/"
#Get ready cascade
face_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_frontalface_alt_tree.xml'))
right_eye_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_righteye_2splits.xml'))
left_eye_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_lefteye_2splits.xml'))
start = time.time()
try :
while True:
if not cap.isOpened():
break
ret, frame = cap.read()
img_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# 1.The face is in the frame
face_points = face_cascade.detectMultiScale(img_gray)
for (fx,fy,fw,fh) in face_points:
#2. ROI(Region of Interest:Target area)Cut out the image that becomes
#Divided into right area and left area respectively(Baron Ashura method)
width_center = fx + int(fw * 0.5)
face_right_gray = img_gray[fy:fy+fh, fx:width_center]
face_left_gray = img_gray[fy:fy+fh, width_center:fx+fw]
#3.Determine if both right and left eyes are visible
right_eye_points = right_eye_cascade.detectMultiScale(face_right_gray)
left_eye_points = left_eye_cascade.detectMultiScale(face_left_gray)
if 0 < len(right_eye_points) and 0 < len(left_eye_points):
(rx,ry,rw,rh) = right_eye_points[0]
(lx,ly,lw,lh) = left_eye_points[0]
#4.Surround the face, right eye and left eye with a rectangle
#Right eye is orange
cv2.rectangle(frame,(fx+rx,fy+ry),(fx+rx+rw,fy+ry+rh),(0,255,255),2)
#Left eye is red
cv2.rectangle(frame,(width_center+lx,fy+ly),(width_center+lx+lw,fy+ly+lh),(0,0,255),2)
#The whole face is green
cv2.rectangle(frame,(fx,fy),(fx+fw,fy+fh),(0,255,0),2)
writer.write(frame)
except cv2.error as e:
print(e)
print("processing time{}Seconds".format(time.time() - start))
writer.release()
cap.release()
For comparison, I will also list the one using both eyes haarcascade_eye.xml.
Both eyes version
import cv2
import os
import time
if __name__ == '__main__':
cap = cv2.VideoCapture('one_minutes.mp4')
cap_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
cap_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = cap.get(cv2.CAP_PROP_FPS)
fourcc = cv2.VideoWriter_fourcc('m','p','4','v')
writer = cv2.VideoWriter('detect_face_2.mp4',fourcc, fps, (cap_width, cap_height))
cascade_base_path = "/usr/local/Cellar/opencv/4.1.1_2/share/opencv4/haarcascades/"
face_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_frontalface_alt_tree.xml'))
eyes_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_eye.xml'))
start = time.time()
try :
while True:
if not cap.isOpened():
break
ret, frame = cap.read()
img_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
face_points = face_cascade.detectMultiScale(img_gray)
for (fx,fy,fw,fh) in face_points:
face_gray = img_gray[fy:fy+fh, fx:fx+fw]
eyes_points = eyes_cascade.detectMultiScale(face_gray)
if 0 < len(eyes_points):
for (ex,ey,ew,eh) in eyes_points:
#Eyes are orange
cv2.rectangle(frame,(fx+ex,fy+ey),(fx+ex+ew,fy+ey+eh),(0,255,255),2)
#The whole face is green
cv2.rectangle(frame,(fx,fy),(fx+fw,fy+fh),(0,255,0),2)
writer.write(frame)
except cv2.error as e:
print(e)
print("processing time{}Seconds".format(time.time() - start))
writer.release()
cap.release()
result
What I found
――The accuracy was higher when the right eye and the left eye were judged separately. Especially when I saw the competition, I recognized it more than both eyes! -Although it is recognition of both eyes, it will be misrecognized other than the eyes ――Even if your face is tilted a little, the accuracy will drop sharply. I haven't broken N ○ H: tired_face: It doesn't seem to recognize it without a cascade that trains tilted and resized images. It may be best to use Deep Learning now. Certainly, I think that the accuracy seems to be good because I am learning after duplicating one piece of data in the library by tilting, resizing, blurring, etc .: thinking: (I have to remember new things again. Is it ...: weary: I don't have the energy and physical strength: thermometer_face :) --The processing time took 276.574 seconds. (About 4 minutes) If it is C ++, it will end soon.
in conclusion
A few days ago, Try real-time face detection with OpenCV was posted and was dropped: scream: It took more time to put together than development: weary:
Referenced link
-[Face recognition using OpenCV (Haar-like feature classifier)](https://qiita.com/hitomatagi/items/04b1b26c1bc2e8081427#haar-like%E7%89%B9%E5%BE%B4%E5% 88% 86% E9% A1% 9E% E5% 99% A8) -[Explanation for beginners] openCV face detection mechanism and practice (detectMultiScale) --Detailed explanation. It seems that the accuracy is not good if the object you want to track is rotating
Recommended Posts