Try to find mistakes using OpenCV and Python.

Preparation

First, make an image for finding mistakes. It's hard to make, so this time, I will borrow the image of the web page Brain Training Spot the Difference Illustration 01 (Beginner). ..

And since this image is vertically connected to A image and B image, it is divided into upper and lower parts. In addition to dividing it into two, convert it to PNG format for handling with OpenCV.

convert -crop 100%x50% image02.gif image02.png

This will create the upper and lower split files iamge02-0.png and image02-1.png, and you are ready to go.

Since the label is written in the image as "A" in the upper image and "B" in the lower image, the file names are also changed to image02A.png and image02B.png.

`convert image02.gif image02.You can png it and split it with python.`



```python
import cv2

img = cv2.imread('img/image02.png')
height, width, _ = img.shape
imageA = img[0:round(height/2), :]
imageB = img[round(height/2):, :]
cv2.imwrite('img/image02A.png', imageA)
cv2.imwrite('img/image02B.png', imageB)

Spot the difference 1

Load A image and B image and try feature detection.

def matchAB(fileA, fileB):
    #Loading images
    imgA = cv2.imread(fileA)
    imgB = cv2.imread(fileB)

    #Gray conversion
    grayA = cv2.cvtColor(imgA, cv2.COLOR_BGR2GRAY)
    grayB = cv2.cvtColor(imgB, cv2.COLOR_BGR2GRAY)

    #Extraction of AKAZE features
    akaze = cv2.AKAZE_create()
    kpA, desA = akaze.detectAndCompute(grayA, None)
    kpB, desB = akaze.detectAndCompute(grayB, None)

    #BFMatcher definition and imaging
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
    matches = bf.match(desB, desB)
    matches = sorted(matches, key=lambda x: x.distance)
    matched_image = cv2.drawMatches(imgA, kpA, imgB, kpB, matches, None, flags=2)
    
    #display
    plt.imshow(cv2.cvtColor(matched_image, cv2.COLOR_BGR2RGB))
    plt.show()

Obviously, the above method would mark "matching places", but this is not good because there are only 5 wrong places.

Find the difference 2

What is spot the difference in the first place? In other words, when two images are superimposed, the outline of the picture drawn on each may be slightly different.

Considering the possibility that image A and image B are slightly out of position, we decided to find the position where the window created by dividing image A best matches image B and create a difference image. did.

def matchAB(fileA, fileB):
    #Loading images
    imgA = cv2.imread(fileA)
    imgB = cv2.imread(fileB)

    #Gray conversion
    grayA = cv2.cvtColor(imgA, cv2.COLOR_BGR2GRAY)
    grayB = cv2.cvtColor(imgB, cv2.COLOR_BGR2GRAY)

    #Get image size
    height, width = grayA.shape
    #Create a partial image and match it
    result_window = np.zeros((height, width), dtype=imgA.dtype)
    for start_y in range(0, height-100, 50):
        for start_x in range(0, width-100, 50):
            window = grayA[start_y:start_y+100, start_x:start_x+100]
            match = cv2.matchTemplate(grayB, window, cv2.TM_CCOEFF_NORMED)
            _, _, _, max_loc = cv2.minMaxLoc(match)
            matched_window = grayB[max_loc[1]:max_loc[1]+100, max_loc[0]:max_loc[0]+100]
            result = cv2.absdiff(window, matched_window)
            result_window[start_y:start_y+100, start_x:start_x+100] = result

	plt.imshow(result_window)

With this, it turned out that there was a position where the difference was somehow large.

Continuation of Finding Differences 2

So, this time, I will pick up the coordinates of the "position where there is a difference" in the difference image and try to superimpose it on the original image. Specifically, the coordinate values are picked up by extracting the outline of the "position with a difference", and a square is drawn on the original image based on the coordinates.

def matchAB(fileA, fileB):
    #Loading images
    imgA = cv2.imread(fileA)
    imgB = cv2.imread(fileB)

    #Gray conversion
    grayA = cv2.cvtColor(imgA, cv2.COLOR_BGR2GRAY)
    grayB = cv2.cvtColor(imgB, cv2.COLOR_BGR2GRAY)

    #Get image size
    height, width = grayA.shape
    #Create a partial image and match it
    result_window = np.zeros((height, width), dtype=imgA.dtype)
    for start_y in range(0, height-100, 50):
        for start_x in range(0, width-100, 50):
            window = grayA[start_y:start_y+100, start_x:start_x+100]
            match = cv2.matchTemplate(grayB, window, cv2.TM_CCOEFF_NORMED)
            _, _, _, max_loc = cv2.minMaxLoc(match)
            matched_window = grayB[max_loc[1]:max_loc[1]+100, max_loc[0]:max_loc[0]+100]
            result = cv2.absdiff(window, matched_window)
            result_window[start_y:start_y+100, start_x:start_x+100] = result

    #Extract the outline of the difference image created as a result of matching and enclose it in a square
    _, result_window_bin = cv2.threshold(result_window, 127, 255, cv2.THRESH_BINARY)
    _, contours, _ = cv2.findContours(result_window_bin, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    imgC = imgA.copy()
    for contour in contours:
        min = np.nanmin(contour, 0)
        max = np.nanmax(contour, 0)
        loc1 = (min[0][0], min[0][1])
        loc2 = (max[0][0], max[0][1])
        cv2.rectangle(imgC, loc1, loc2, 255, 2)

    #Display image
    plt.subplot(1, 3, 1), plt.imshow(cv2.cvtColor(imgA, cv2.COLOR_BGR2RGB)), plt.title('A'), plt.xticks([]), plt.yticks([])
    plt.subplot(1, 3, 2), plt.imshow(cv2.cvtColor(imgB, cv2.COLOR_BGR2RGB)), plt.title('B'), plt.xticks([]), plt.yticks([])
    plt.subplot(1, 3, 3), plt.imshow(cv2.cvtColor(imgC, cv2.COLOR_BGR2RGB)), plt.title('Answer'), plt.xticks([]), plt.yticks([])
    plt.show()

The picture is small and hard to see, but it seems that 5 mistakes can be suggested except for the part of the label with A or B on the upper left.

Summary

It turned out that there is a possibility to solve the spot the difference quiz by creating a difference image using OpenCV.

Today's code

https://github.com/fukuit/Python_SelfLearning/blob/master/feature_detect/feature_detection2.py

[PYTHON] Let's compare the two pictures and look for mistakes