[PYTHON] I tried face detection with MTCNN

In machine learning for face detection, I used only OpenCV, including model creation, so I decided to use others as well. So I tried using MTCNN.

MTCNN seems to use tensorflow behind the scenes. It feels good to be in deep running with one foot.

Preparation

pip3 install cv2
pip3 install matplotlib
pip3 install mtcnn
pip3 install tensorflow

Image used in the test

It's just right for verification, isn't it? Various exposed faces. There are many things other than the face

Face detection with MTCNN

verification code

# face detection with mtcnn on a photograph
from matplotlib import pyplot
from matplotlib.patches import Rectangle
from matplotlib.patches import Circle
from mtcnn.mtcnn import MTCNN
import cv2

# draw an image with detected objects
def draw_image_with_boxes(filename, result_list):
	# load the image
	data = pyplot.imread(filename)
	# plot the image
	pyplot.imshow(data)
	# get the context for drawing boxes
	ax = pyplot.gca()
	# plot each box
	for result in result_list:
		# get coordinates
		x, y, width, height = result['box']
		# create the shape
		rect = Rectangle((x, y), width, height, fill=False, color='red')
		# draw the box
		ax.add_patch(rect)
		# draw the dots
		for key, value in result['keypoints'].items():
			# create and draw dot
			dot = Circle(value, radius=2, color='red')
			ax.add_patch(dot)
	# show the plot
	pyplot.show()
 
filename = 'test.png'
# load image from file
pixels = cv2.cvtColor(cv2.imread(filename), cv2.COLOR_BGR2RGB)
# create the detector, using default weights
detector = MTCNN()
# detect faces in the image
faces = detector.detect_faces(pixels)
# display faces on the original image
draw_image_with_boxes(filename, faces)

result

Everything is detected correctly. It seems that this image has not been falsely detected.

By the way, here is the one using OpenCV (haarcascade_frontalface_default). I can detect all the faces, but they are erroneously detected. Ice cream is really a face w

Here's what I made before because I wasn't satisfied with the default model. False detections are no longer detected, but one person cannot be detected.

Reference) A story about a gopher touching OpenCV https://speakerdeck.com/usk81/introduction-about-opencv-for-gophers

MTCNN ... Completely defeated ... It took about 40 hours ＿|￣|○ il||li

I also tried cutting out

I tried to save only one that can be cut out the largest.

code

# face detection with mtcnn on a photograph
from matplotlib import pyplot
from mtcnn.mtcnn import MTCNN
import cv2

# draw each face separately
def save_biggest_face_image(filename, result_list):
	# load the image
	# data = pyplot.imread(filename)
	data = cv2.imread(filename)
	# plot each face as a subplot
	maxlen = -1
	for i in range(len(result_list)):
		# get coordinates
		x1, y1, width, height = result_list[i]['box']
		if width > height:
			length = width 
		else:
			length = height
		x2, y2 = x1 + length, y1 + length
		# # define subplot
		# pyplot.subplot(1, len(result_list), i+1)
		# pyplot.axis('off')
		# # plot face
		# pyplot.imshow(data[y1:y2, x1:x2])
		d = data[y1:y2, x1:x2]
		if length > 100 and length > maxlen:
			maxlen = length
			md = d
		# cv2.imwrite('/Users/komatsu/Desktop/'+str(i)+'.png', d)

	# show the plot
	# pyplot.show()
	if maxlen > 0:
		cv2.imwrite('result.png', md)
 
filename = 'test.png'
# load image from file
pixels = cv2.cvtColor(cv2.imread(filename), cv2.COLOR_BGR2RGB)
# create the detector, using default weights
detector = MTCNN()
# detect faces in the image
faces = detector.detect_faces(pixels)
# save face image
save_biggest_face_image(filename, faces)

result

Remarks

The mystery is that the image file could not be read by pyplot.imread. I can use it in a function ...