[PYTHON] I tried face recognition using Face ++

Execution environment

Ubuntu 16.04 Python 3.7 opencv-python 4.1.0.25

Overview

Try face recognition using the API "Face ++" developed by China. The procedure is as follows.

** ① Account registration ② Get the API to use ③ Confirmation of demo ④ Program ⑤ Face recognition **

** ① Account registration **

First, you need to register an account to use Face ++. Register on the official page and get your own ** API Key ** and ** API Secret **. These are required when using each API.

You can check the created API Key and API Secret with Apps / API Key of Face ++. a.png

** ② Get the API to use **

There are three types of APIs used in the face recognition program created this time. Both exist in a group called Facial Recognition, Use ** Detect API ** and ** Search API ** to identify faces, and ** FaceSet Create API ** to register faces. You can check the parameters of each API in the official API reference. b.png This time, we will use the API in the red frame.

** ③ Confirm demo **

There is a demo on the official Face ++ page for Search API, so let's see what kind of output you can get. Screenshot from 2019-11-13 10-34-50.png

** Probe Face </ font> ** in the above figure is the comparison source image. ** Candidate Face </ font> ** is selected from the images in the database in descending order of similarity. (In the face recognition program to be created, Probe Face </ font> is used as an unknown input image. Prepare Candidate Face </ font> as an image of the person you want to recognize. )

The comparison result of each image is output as a JSON file.

result.json


{
  "time_used": 420,
  "thresholds": {
    "1e-3": 62.327,
    "1e-5": 73.975,
    "1e-4": 69.101
  },
  "faces": [
    {
      "face_token": "8d25b53edf9e20bad18bebe3b0c8e06c",
      "face_rectangle": {
        "width": 252,
        "top": 170,
        "height": 252,
        "left": 102
      }
    }
  ],
  "results": [
    {
      "confidence": 97.076,
      "user_id": "",
      "face_token": "8323ce719cb1129e4abf4ada1129cbc9"
    },
    {
      "confidence": 93.254,
      "user_id": "",
      "face_token": "fd6c81b63615b62b8506f33a6748fd95"
    },
    {
      "confidence": 66.15,
      "user_id": "",
      "face_token": "192672385b603e6b54cf884cd019a620"
    },
    {
      "confidence": 63.826,
      "user_id": "",
      "face_token": "34bbf05899b53968dcee620aa06a35e7"
    },
    {
      "confidence": 57.875,
      "user_id": "",
      "face_token": "ee6cbca281f449d3ed6040ca63b4c52c"
    }
  ],
  "image_id": "+b1mqy/4tPkV6QMTyRVGyA==",
  "request_id": "1573608536,e6ac4f2a-62fb-40a9-890b-696eba9a32a1"
}

Each element of the results key represents the comparison result between Probe Face </ font> and Candidate Face </ font>. The similarity between the two images is output as ** "confidence" **. Looking at the above results, the confidence of Candidate Face1 </ font> is 97.076, The confidence of Candidate Face2 </ font> is 93.254, which is a high degree of similarity. Other results are below similarity 70. Create a face recognition program using this similarity.

④ Program

The face recognition program is as follows. For how to use the API, refer to the URL below. https://github.com/Doarakko/api-challenge/tree/master/facepp-api

main.py



import time
import re
import base64
import requests
import glob
import cv2

API_KEY = 'My API Key'
API_SECRET = 'My API Secret'

#API to detect faces
def detect_image(img):
    endpoint = 'https://api-us.faceplusplus.com'
    response = requests.post(
        endpoint + '/facepp/v3/detect',
        {
            'api_key': API_KEY,
            'api_secret': API_SECRET,
            'image_base64': img,
        }
    )
    #1 second sleep
    time.sleep(1)
    #If the status code of the response is other than 200
    if response.status_code != 200:
        print('[Error] ')
        return -1
    resources = response.json()
    return resources

#Face identification API
def search_image(img,face_set):
    endpoint = 'https://api-us.faceplusplus.com'
    res, dst_data = cv2.imencode('.jpg', img)
    img_base64 = base64.b64encode(dst_data)
    #Send a request to the face detection API
    faces=detect_image(img_base64)
    for face in faces["faces"]:
        try:
            response = requests.post(
                endpoint + '/facepp/v3/search',
                {
                    'api_key': API_KEY,
                    'api_secret': API_SECRET,
                    'face_token': str(face["face_token"]),
                    'faceset_token': face_set["faceset_token"],
                    'return_result_count': 1,
                }
            )
            #1 second sleep
            time.sleep(1)
            #If the status code of the response is other than 200
            if response.status_code != 200:
                return -1
            resources = response.json()
            print("success")
            print(resources)
            img=draw_img(face,resources,img)
        except Exception as e:
            return -1
    #Save image
    cv2.imwrite("./output/result_"+filename, img)
    return resources

#Draw face information with the highest probability
def draw_img(face,resources,img):
    try:
        left = face["face_rectangle"]["left"]
        top = face["face_rectangle"]["top"]
        right = left + face["face_rectangle"]["width"]
        bottom = top + face["face_rectangle"]["height"]
        #Draw a rectangle
        cv2.rectangle(img, (left, top), (right, bottom), (0, 0, 255), 3)
        font = cv2.FONT_HERSHEY_SIMPLEX
        #Probability drawing
        cv2.putText(img, str(resources["results"][0]["confidence"]) + "%", (left, top), font, 0.5, (255, 255, 255), 3, cv2.LINE_AA)
        cv2.putText(img, str(resources["results"][0]["confidence"]) + "%", (left, top), font, 0.5, (0, 0, 0), 1, cv2.LINE_AA)
        #Drawing the name
        cv2.putText(img, str(resources["results"][0]["user_id"]) , (left, top-20), font, 0.5, (255, 255, 255), 3, cv2.LINE_AA)
        cv2.putText(img, str(resources["results"][0]["user_id"]) , (left, top-20), font, 0.5, (0, 0, 0), 1, cv2.LINE_AA)
    except Exception as e:
        print("none")

    return img

#Register face images to compare
def create_faceset(face_list,tags):
    id_list = []
    for (img_path,face_id) in zip(face_list,tags):
        img_file = base64.encodebytes(open(img_path, 'rb').read())
        resources = detect_image(img_file)
        set_userid(resources["faces"][0]["face_token"],face_id)
        id_list.append(resources["faces"][0]["face_token"])

    id_list=str(id_list).replace("'", "")
    id_list=str(id_list).replace("[", "")
    id_list=str(id_list).replace("]", "")
    id_list=str(id_list).replace(" ", "")

    print(resources)
    print(id_list)

    endpoint = 'https://api-us.faceplusplus.com'
    try:
        response = requests.post(
            endpoint + '/facepp/v3/faceset/create',
            {
                'api_key': API_KEY,
                'api_secret': API_SECRET,
                'display_name': 'facebank',
                'face_tokens': id_list,
            }
        )
        #1 second sleep
        time.sleep(1)
        #If the status code of the response is other than 200
        if response.status_code != 200:
            return -1
        resources = response.json()
        print(resources)
        return resources
    except Exception as e:
        return -1

def set_userid(face_token,user):
    endpoint = 'https://api-us.faceplusplus.com'
    try:
        response = requests.post(
            endpoint + '/facepp/v3/face/setuserid',
            {
                'api_key': API_KEY,
                'api_secret': API_SECRET,
                'display_name': 'facebank',
                'face_token':face_token,
                'user_id':user,
        }
        )
        #1 second sleep
        time.sleep(1)
        #If the status code of the response is other than 200
        if response.status_code != 200:
            return -1
        resources = response.json()
        print(resources)
        return resources
    except Exception as e:
        return -1

if __name__ == '__main__':
    #Get the image you want to identify
    filename="input.jpg "
    img=cv2.imread('./input/'+filename)
    #Get the images in the folder
    face_list = glob.glob('./facebank/*/*.jpg')
    face_list.sort()
    tags=[]#Each name(user_id)For storage
    #Read from registration data
    for face_path in face_list:
        tags.append(str(re.search(r'./facebank/(.+)/.+', face_path).group(1)))
    #Define faces to compare with API
    face_set = create_faceset(face_list,tags)
    #Send a request to the identifying API
    resources = search_image(img,face_set)

For the arguments of each API, refer to Official Reference.

The folder hierarchy is as follows.

─ facebank/
    ├ A/ 
    │ └ A.jpg
    ├ B/ 
    │ └ B.jpg
    ├ C/ 
    │ └ C.jpg
    ├ D/ 
    │ └ D.jpg
    └ E/
      └ E.jpg
─ input/
    └ input.jpg
─ output/
    └ result_input.jpg
─ main.py

Put the image to identify in the input folder. Prepare a folder with each person's name in the facebank folder, Put your face photo in that folder. (The name of the face photo is arbitrary)

⑤ Face recognition

Face recognition. The photos to be registered and recognized were borrowed from "Pakutaso", which publishes free materials of people. The following 5 photos are registered as recognition targets this time.

--ezaki ezakisan --Murata muratasan --narishige muratasan --okawa muratasan --yusei muratasan

And here is the recognition result! result.jpg result_helpIMGL6330_TP_V.jpg result_PAK86_sotugyousei15213941_TP_V.jpg result_N612_shibuyagawadedenwasuruikemen_TP_V.jpg

All the people who registered their faces are correct with a similarity of 80% or more! Even though the number of registered face photos is one for each person, it is highly accurate.

People with unregistered faces are also identified as having the highest degree of similarity among registered people, It can be seen that the degree of similarity is low as shown in the figure below. 333.jpg 222.jpg

If you register an account with Face ++, you can perform face recognition with your own image on the Demo on the WEB page. If you are interested, why not give it a try.

Recommended Posts