Feature matching with OpenCV 3 and Python 3 (A-KAZE, KNN)

Feature matching with OpenCV

Feature matching is the matching of features extracted from different images.

• Create panoramic image
• Object detection
• Motion tracking

It is a technology that appears in.

The following libraries are provided in OpenCV.

• Feature extraction
• Feature matching
• Display matching results

This time, using OpenCV 3 + Python 3, we will try to match the features of the rotated and zoomed image as shown below.

** Draw matching result **

What is OpenCV

OpenCV (Open Source Computer Vision Library) is a collection of BSD-licensed video / image processing libraries. There are many algorithms for image filtering, template matching, object recognition, video analysis, machine learning, and more.

■ Example of motion tracking using OpenCV (OpenCV Google Summer of Code 2015) https://www.youtube.com/watch?v=OUbUFn71S4s

■ Click here for installation and easy usage Install OpenCV 3 (core + contrib) in Python 3 environment & Difference between OpenCV 2 and OpenCV 3 & simple operation check

■ Click here for still image processing Try edge detection with OpenCV Perform various filters with OpenCV (gradient, high-pass, Laplacian, Gaussian) Extract feature points with OpenCV (AgastFeature, FAST, GFTT, MSER, AKAZE, BRISK, KAZE, ORB, SimpleBlob) Face recognition using OpenCV (Haar-like feature classifier) Estimate who's face using OpenCV (Eigenface, Fisherface, LBPH) Recognize the contour and direction of a shaped object with OpenCV (principal component analysis: PCA, eigenvectors)

■ Click here for video processing Try converting videos in real time with OpenCV Try converting webcam / camcorder video in real time with OpenCV Draw optical flow in real time with OpenCV (Shi-Tomasi method, Lucas-Kanade method) Object tracking using OpenCV (tracking feature points specified by mouse by Lucas-Kanade method Motion template analysis using OpenCV (recognizes objects and their moving directions in real time)

A-KAZE

• Homepage 　http://www.robesafe.com/personal/pablo.alcantarilla/kaze.html
• License 　https://github.com/pablofdezalc/akaze/blob/master/LICENSE
• Features In OpenCV 3, the patented SIFT (Scale-invariant feature transform) and SURF (Speed-Upped Robust Feature) have been moved to the contrib library, and the license-friendly KAZE and A-KAZE have been moved to the core library. It is now included. A-KAZE (ACCELERATED-KAZE) is an algorithm that improves KAZE at high speed. KAZE, A-KAZE is a robust feature extraction algorithm that is resistant to scaling, rotation, and lighting changes, similar to SIFT and SURF. SIFT and SURF are weak in extracting features in flat areas because they are algorithms that use Gaussian filters, but KAZE and A-KAZE also extract features in flat areas.
• Comparison of ORB and A-KAZE Similar to SIFT and SURF, it is robust in scaling and rotation, but has the same tracking performance as ORB (Oriented-BRIEF), which is high speed (10 times SURF and 100 times SIFT). 　
  ** ORB vs A-KAZE (youtube link) ** A-KAZE seems to track features in flat areas better than ORB.

KNN、Brute-Force、FLANN KNN (K-Nearest Neighbor algorithm) is an algorithm that selects K nearest neighbor labels from the search space and assigns class labels by majority vote. Learning simply stores the training data as it is. It operates at high speed because the learning cost is zero. A representative player of the laziness learning algorithm. OpenCV supports the brute-force method (Brute-Force) and the fast approximate neighbor search method (FLANN) as a method for searching K nearest neighbor labels from the search space.

• Brute-Force It takes time because it is a brute force search that searches the entire search space, but it is possible to reliably search for the nearest feature quantity.
• Fast Library for Approximate Nearest Neighbors (FLANN) Only the space close to the feature point to be searched is searched. Since the search space is reduced, you can search at high speed. However, if the wrong search parameter is specified, the search space will be selected incorrectly, and only the search space that does not include the nearest feature points will be searched. I will end up.

● Remarks (Python3 and FLANN)

Earlier I wrote that there is a tutorial that doesn't work on OpenCV 3 (link), but FLANN feature point matching is my environment (OpenCV) It didn't work on 3.1.0 + Python 3.5.2 + Windows 10).

    > matches = flann.knnMatch(des1,des2,k=2)
    > 

\ # The following error occurs # error: C:\dev\opencv-3.1.0\modules\python\src2\cv2.cpp:163: error: (-215) The data should normally be NULL! in function NumpyAllocator::allocate

FLANN works fine in C ++ and OpenCV2, so if you want to use FLANN, run it in C ++ or OpenCV2 environment. When working with a combination of OpenCV 3 and Python 3, currently the "brute-force method" is used.

program

• Operating environment
  • OpenCV 3.1.0
  • Python 3.5.2
• Sample image I prepared the images taken separately (img1.jpg, img2.jpg). The image includes rotation and zoom.

knn.py

# -*- coding: utf-8 -*-
import cv2

#Image 1
img1 = cv2.imread("img1.jpg ")
#Image 2
img2 = cv2.imread("img2.jpg ")

# A-Generation of KAZE detector
akaze = cv2.AKAZE_create()                                

#Feature detection and feature vector calculation
kp1, des1 = akaze.detectAndCompute(img1, None)
kp2, des2 = akaze.detectAndCompute(img2, None)

# Brute-Force Matcher generation
bf = cv2.BFMatcher()

#Brute feature vectors-Matching with Force & KNN
matches = bf.knnMatch(des1, des2, k=2)

#Thin out data
ratio = 0.5
good = []
for m, n in matches:
    if m.distance < ratio * n.distance:
        good.append([m])

#Draw corresponding feature points
img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, good[], None, flags=2)

#Image display
cv2.imshow('img', img3)

#Press the key to finish
cv2.waitKey(0)
cv2.destroyAllWindows()

Matching result

The Ratio Test proposed by D. Lowe thins out the matching results and displays them.

lowe.py

#Thin out data
ratio = 0.5
lowe = []
for m, n in matches:
    if m.distance < ratio * n.distance:
        lowe.append([m])

#Draw corresponding feature points
img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, lowe[], None, flags=2)

This method gives you a visual idea of the matching status of the entire image. There seems to be no false positives in this test.

Let's change a part of the code so that only those with good matching status between features are displayed.

knn_good.py

#Sort features according to matching status
good = sorted(matches, key = lambda x : x[1].distance)

#Draw corresponding feature points
img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, good[:30], None, flags=2)

A different feature than before was drawn. This time, since the features are dispersed, they match as a whole, but in the case of an image in which the feature points are partially biased, those with good matching conditions will be concentrated there. ..

KeyPoint (feature point object)

DMatch (matching result object)

The following items are stored in DMatch as a result of matching between features.