[PYTHON] I tried to sort out the objects from the image of the steak set meal-② Overlap number sorting

Introduction

Last time, Selective Search was used to detect objects in the image of the steak set meal. Although the accuracy has improved, I noticed that out of about 50 images, they are classified into about 3 types: dust-like images, the same images (with slightly different cropping positions), and necessary object images. .. This time, I will try to see if I can select only the necessary object images from them.

hypothesis

Isn't an image with a large number of overlaps necessary when extracting an object from an image? However, in reality, it is possible to exclude the rectangular child part of the parent-child relationship.

What is this guy saying

When detecting an object from an image, it is extracted as a rectangle, but the part where the rectangles are mixed is called the overlap. The red part of the image below.

Reference source

-(Fun hit detection course 1-Hit judgment between rectangles (beginner)-) [http://d.hatena.ne.jp/ono36/20070718/p1]

Source code

I wrote it for the time being.

group_image

# -*- coding: utf-8 -*-

import cv2
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import selectivesearch
import os

def main():
    # loading lena image
    img = cv2.imread("{Steak set meal image}")

    # perform selective search
    img_lbl, regions = selectivesearch.selective_search(
        img,
        scale=500,
        sigma=0.9,
        min_size=10
    )

    candidates = set()

    for r in regions:
        # excluding same rectangle (with different segments)
        if r['rect'] in candidates:
            continue

        # excluding regions smaller than 2000 pixels
        if r['size'] < 2000:
            continue

        # distorted rects
        x, y, w, h = r['rect']

        if w / h > 1.2 or h / w > 1.2:
            continue

        candidates.add(r['rect'])

    # draw rectangles on the original image
    fig, ax = plt.subplots(ncols=1, nrows=1, figsize=(6, 6))
    ax.imshow(img)

    overlaps = {}

    #Count the number of overlaps and assign them to the array.
    for x, y, w, h in candidates:
        group = '%s_%s_%s_%s' % (x, y, w, h)

        for x2, y2, w2, h2 in candidates:
            if x2 - w < x < x2 + w2 and y2 - h < y < y2 + h2:

                if not group in overlaps:
                    overlaps[group] = 0

                overlaps[group] = overlaps[group] + 1

    print overlaps

    #Outputs files with 30 or more overlaps (30 is arbitrarily thresholded).
    for key, overlap in enumerate(overlaps):
        if overlap > 30:
            for x, y, w, h in candidates:
                group = x + y + w + h

                if group in overlaps:
                    cv2.imwrite("{Directory path}" + str(group) + '.jpg', img[y:y + h, x:x + w])

Supplement

--Object Determines if it overlaps with the detected image.

if x2 - w < x < x2 + w2 and y2 - h < y < y2 + h2:

--Only the extraction results with 30 or more overlaps are saved as images.

result

(Original) 50 sheets → 36 sheets

About 30% of the images of the steak set meal have been removed.

In addition, 5 types of object images of previous remained.

Summary

――This time, since there is only one type of verification image, we have made some adjustments and result judgments, so if you try with other images, you may get different results. ――Next time, I would like to do something like clustering.

All page links

-I tried object detection using Python and OpenCV -I tried to sort out objects from the image of steak set meal-① Object detection -I tried to sort out the objects from the image of the steak set meal-② Overlap number sorting -I tried to sort out the objects from the image of the steak set meal-③ Similar image heat map detection -I tried to sort out the objects from the image of the steak set meal-④ Clustering -I tried to sort out objects from the image of steak set meal-⑤ Similar image feature point detection edition