Recognize the contour and direction of a shaped object with OpenCV3 and Python3 (Principal component analysis: PCA, eigenvectors)

What is Principal Component Analysis (PCA)?

There are various features of an object, and it may be possible to identify the type of object by the features. For example, coffee has features such as sourness, bitterness, roasting, richness, color, and aroma, and the combination can identify the type of coffee (mocha, kirimanjaro, mandelin, etc.). .. However, there are various features, and when identifying the type, some features are effective and some are not very effective. For example, weight and hardness can be considered as features of coffee, but this is unlikely to be very useful for identifying the type. The price may be a little useful as a feature.

Principal component analysis (PCA) is an analysis method that uses standard deviation, covariance matrix, eigenvectors, etc. to identify strongly characteristic components when identifying an object. In addition, by paying attention to the features that are closely related, it is possible to perform dimensional compression of the data, which is effective in reducing the amount of calculation and the amount of data / memory.

OpenCV also supports Principal Component Analysis (PCA) in the core module. This time, I will create a program that calculates the direction of a shaped object using a method that handles Principal Component Analysis (PCA) of OpenCV and displays the outline and the direction of the principal component with arrows. The final result should look like the figure below.

pca_output.png ** Display the outline (red) and principal component vector (light blue) of the object **

OpenCV OpenCV (Open Source Computer Vision Library) is a collection of BSD-licensed video / image processing libraries. There are many algorithms for image filtering, template matching, object recognition, video analysis, machine learning, and more.

■ Example of motion tracking using OpenCV (OpenCV Google Summer of Code 2015) https://www.youtube.com/watch?v=OUbUFn71S4s

■ Click here for installation and easy usage Install OpenCV 3 (core + contrib) in Python 3 environment & Difference between OpenCV 2 and OpenCV 3 & simple operation check

■ Click here for still image processing Try edge detection with OpenCV Perform various filters with OpenCV (gradient, high-pass, Laplacian, Gaussian) Extract feature points with OpenCV (AgastFeature, FAST, GFTT, MSER, AKAZE, BRISK, KAZE, ORB, SimpleBlob) Face recognition using OpenCV (Haar-like feature classifier) Estimate who's face using OpenCV (Eigenface, Fisherface, LBPH)

■ Click here for video processing Try converting videos in real time with OpenCV Try converting webcam / camcorder video in real time with OpenCV Draw optical flow in real time with OpenCV (Shi-Tomasi method, Lucas-Kanade method) Object tracking using OpenCV (tracking feature points specified by mouse by Lucas-Kanade method Motion template analysis using OpenCV (recognizes objects and their moving directions in real time)

algorithm

Usually, when it comes to principal component analysis, you often see the following points in the scatter plot. The plot is a two-dimensional compression of features that were originally multidimensional. In addition, the arrows represent the vectors of the first and second principal components for the plot. GaussianScatterPCA.svg.png Source: Wikipedia-Principal component analysis

Principal component analysis of figures can also be performed with this application. The procedure is as follows.

  1. Converts color information that seems unnecessary to recognize the direction of an object into black and white information.
  2. Recognize the outline of the object as a two-dimensional (x-coordinate, y-coordinate) point.
  3. Principal component analysis (PCA) is performed on a collection of contour points (x, y).
  4. Find the vector (eigenvector) in the direction of the principal component and draw it on the image.
  5. Repeat steps 1 to 4 for each object.

So, if you use OpenCV, you can recognize the outline and direction of an object just by calling a few methods.

program

--Instructed to acquire only one dimension of the main component (maxComponents = 1) --Instructed to acquire the data of the midpoint because the coordinates of the contour are used for the principal component analysis (cv2.CHAIN_APPROX_NONE). --Usually, when performing principal component analysis, data is standardized as preprocessing. In this image processing, there is no difference in the weighting of the two-dimensional x-coordinate and y-coordinate, so no pre-processing for data standardization is performed.

pca.py


# -*- coding: utf-8 -*-
import cv2
import math
import numpy as np


#Draw vector
def drawAxis(img, start_pt, vec, colour, length):
    #Antialiasing
    CV_AA = 16

    #End point
    end_pt = (int(start_pt[0] + length * vec[0]), int(start_pt[1] + length * vec[1]))

    #Draw center
    cv2.circle(img, (int(start_pt[0]), int(start_pt[1])), 5, colour, 1)

    #Draw axis
    cv2.line(img, (int(start_pt[0]), int(start_pt[1])), end_pt, colour, 1, CV_AA);

    #Draw the arrow at the tip
    angle = math.atan2(vec[1], vec[0])
    print(angle)

    qx0 = int(end_pt[0] - 9 * math.cos(angle + math.pi / 4));
    qy0 = int(end_pt[1] - 9 * math.sin(angle + math.pi / 4));
    cv2.line(img, end_pt, (qx0, qy0), colour, 1, CV_AA);

    qx1 = int(end_pt[0] - 9 * math.cos(angle - math.pi / 4));
    qy1 = int(end_pt[1] - 9 * math.sin(angle - math.pi / 4));
    cv2.line(img, end_pt, (qx1, qy1), colour, 1, CV_AA);


if __name__ == '__main__':
    #Load image
    src = cv2.imread("pca_test1.jpg ")
    
    #Convert to grayscale
    gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)

    #Binarization
    retval, bw = cv2.threshold(gray, 50, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)

    #Extract contour
    #   contours : [region][Point No][0][x=0, y=1]
    #   cv2.CHAIN_APPROX_NONE:Hold the midpoint
    #   cv2.CHAIN_APPROX_SIMPLE:Do not hold midpoint
    img, contours, hierarchy = cv2.findContours(bw, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
    
    #Processing for each contour
    for i in range(0, len(contours)):

        #Calculate contour area
        area = cv2.contourArea(contours[i])

        #Exclude noise (too small area) and overall contour (too large area)
        if area < 1e2 or 1e5 < area:
            continue
        
        #Draw contour
        cv2.drawContours(src, contours, i, (0, 0, 255), 2, 8, hierarchy, 0)
        
        #Store contour data in a floating point array
        X = np.array(contours[i], dtype=np.float).reshape((contours[i].shape[0], contours[i].shape[2]))
        
        #PCA (1D)
        mean, eigenvectors = cv2.PCACompute(X, mean=np.array([], dtype=np.float), maxComponents=1)
        
        #Draw vector in direction of principal component
        pt = (mean[0][0], mean[0][1])
        vec = (eigenvectors[0][0], eigenvectors[0][1])
        drawAxis(src, pt, vec, (255, 255, 0), 150)


    #display
    cv2.imshow('output', src)
    cv2.waitKey(0)
    
    #End processing
    cv2.destroyAllWindows()

The execution result will look like the image shown at the beginning.

bonus

Let's recognize the contour and direction not only for the sample image but also for other images. For example, let's analyze by placing a hanger with a characteristic hole on a pattern like a tatami mat. As a result, we were able to display the vector in the direction of the principal component for the three contours surrounding the small hole, the outer contour and the inner contour. hanger_out.png

Recommended Posts

Recognize the contour and direction of a shaped object with OpenCV3 and Python3 (Principal component analysis: PCA, eigenvectors)
[Python] Comparison of Principal Component Analysis Theory and Implementation by Python (PCA, Kernel PCA, 2DPCA)
Challenge principal component analysis of text data with Python
Principal component analysis (PCA) and independent component analysis (ICA) in python
[GWAS] Plot the results of principal component analysis (PCA) by PLINK
Principal component analysis with Power BI + Python
Calculate the shortest route of a graph with Dijkstra's algorithm and Python
Make a DNN-CRF with Chainer and recognize the chord progression of music
Get and estimate the shape of the head using Dlib and OpenCV with python
Try to separate the background and moving object of the video with OpenCV
Dimensional compression with self-encoder and principal component analysis
Robot grip position (Python PCA principal component analysis)
A discussion of the strengths and weaknesses of Python
Get the stock price of a Japanese company with Python and make a graph
Get a large amount of Starbucks Twitter data with python and try data analysis Part 1
[OpenCV / Python] I tried image analysis of cells with OpenCV
Visualize the range of interpolation and extrapolation with python
Mathematical understanding of principal component analysis from the beginning
Calculate the regression coefficient of simple regression analysis with python
Fill the background with a single color with OpenCV2 + Python
Principal component analysis using python from nim with nimpy
Principal component analysis (Principal component analysis: PCA)
A server that returns the number of people in front of the camera with bottle.py and OpenCV
I tried "gamma correction" of the image with Python + OpenCV
Perform isocurrent analysis of open channels with Python and matplotlib
Detect objects of a specific color and size with Python
Create a simple video analysis tool with python wxpython + openCV
Visualize the correlation matrix by principal component analysis in Python
Estimate the attitude of AR markers with Python + OpenCV + drone
Play with the password mechanism of GitHub Webhook and Python
Find the white Christmas rate by prefecture with Python and map it to a map of Japan
A Python beginner first tried a quick and easy analysis of weather data for the last 10 years.
Save the result of the life game as a gif with python
Color extraction with Python + OpenCV solved the mystery of the green background
[python, ruby] fetch the contents of a web page with selenium-webdriver
Introduction to Python Basics of Machine Learning (Unsupervised Learning / Principal Component Analysis)
Create a striped illusion with gamma correction for Python3 and openCV3
How to make a surveillance camera (Security Camera) with Opencv and Python
Make a simple OMR (mark sheet reader) with Python and OpenCV
The story of making a standard driver for db with python.
Draw a watercolor illusion with edge detection in Python3 and openCV3
Solve the Python knapsack problem with a branch and bound method
The idea of feeding the config file with a python file instead of yaml
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
The story of making a module that skips mail with python
[Python] Try to recognize characters from images with OpenCV and pyocr
Create a compatibility judgment program with the random module of python.
Shining life with Python and OpenCV
The story of Python and the story of NaN
Neural network with OpenCV 3 and Python 3
Principal component analysis with Spark ML
Python: Unsupervised Learning: Principal Component Analysis
Coexistence of Python2 and 3 with CircleCI (1.0)
Basic study of OpenCV with Python
I replaced the numerical calculation of Python with Rust and compared the speed
The story of making a university 100 yen breakfast LINE bot with Python
[AtCoder explanation] Control the A, B, C problems of ABC182 with Python!
How to crop the lower right part of the image with Python OpenCV
Get the number of searches with a regular expression. SeleniumBasic VBA Python
The story of having a hard time introducing OpenCV with M1 MAC
[AtCoder explanation] Control the A, B, C problems of ABC186 with Python!