[PYTHON] Computer Vision: Object Detection --Non Maximum Suppression

Target

Implement Non Maximum Suppression used for object detection.

Introduction

Introduced here are the Intersection over Union and Non Maximum Suppression used in Computer Vision: Object Detection Part2-Single Shot Multi Detector.

Non Maximum Suppression (NMS) In object detection, as shown in the figure below, multiple candidate regions that are presumed to have an object may be obtained for one recognized object.

The part surrounded by the colored rectangle points to the candidate area, and the display on the upper left of each candidate area shows the category and certainty of the objects existing in the candidate area.

Non Maximum Suppression is a process that wants to keep only the candidate areas with the highest certainty among the multiple candidate areas obtained, that is, suppresses the ones that are not the maximum.

Now that the purpose of narrowing down the candidate areas has been decided, let's think about how to achieve it.

Intersection over Union (IoU) Non Maximum Suppression performs threshold processing based on Intersection over Union (IoU), which is an index for quantifying the degree of overlap, between the candidate region with the maximum object certainty and other candidate regions.

Considering the two candidate areas as shown in the figure below, IoU can be calculated from the following formula.

IoU = \frac{a \cap b}{a \cup b}

The symbols $ a \ cap b $ and $ a \ cup b $ used in set theory represent the colored areas in the figure below, respectively. When implementing, calculate the IoU by calculating the area of each candidate area, intersection, and total area.

IoU takes a value of [0, 1], the larger the overlap, the closer the value is to 1, and the smaller the overlap, the smaller the IoU value. Therefore, set the threshold value in advance and delete the candidate area with IoU larger than the threshold value. If the IoU value is less than the threshold, it is considered that another object is being detected and the candidate area is left.

Implementation

Execution environment

hardware

-CPU Intel (R) Core (TM) i7-6700 4.00GHz

software

・ Windows 10 Pro 1909 ・ Python 3.6.6 ・ Numpy 1.17.3 ・ Opencv-contrib-python 4.1.2.30

Program to run

The implemented program is published on GitHub.

non_maximum_suppression.py

result

When the threshold was set to 0.1 and run, the many initially displayed candidate areas were narrowed down to one each.

reference

Computer Vision : Object Detection Part2 - Single Shot Multi Detector

Tatsuya Harada. "Machine Learning Professional Series Image Recognition", Kodansha, 2017.