Image Processing Collection in Python

A memorandum of functions of the image processing library often used in Python. Will be updated from time to time.

It is assumed that the following libraries are imported Library Version opencv @3.2.0_1+contrib+python27 scikit-image @0.12.3_2 scipy @0.19.0_0+gfortran

import numpy as np
import copy
import cv2 as cv
import skimage
import skimage.io as sk
import skimage.filters as fil
import skimage.filters.rank as rank
import skimage.morphology as mor
import skimage.transform as tr
import scipy.misc as sp
import scipy.ndimage as ndi

Read Image Comparison by library used to load images

scikit-image

img = sk.imread(filename,as_grey=False,plugin=None,flatten=None)

• Multi-channel image can be read (this is for 3D multi-channel images)
• Grayscale conversion is performed when reading an RGB image with as_gray = True, but you need to be careful because the pixel value will be crushed due to 64bit-float.

OpenCV

img = cv.imread((filename[, flags])

• flags

• cv.IMREAD_COLOR (or 1): Read RGB image
• cv.IMREAD_GRAYSCALE (or 0): Convert to grayscale and read
• cv.IMREAD_UNCHANGED (or -1): Read without conversion (alpha channel can also be read)

• Note that the channel will be BGR when reading RGB images.
• Can be converted to RGB with cv.cvtColor (image, cv2.COLOR_BGR2RGB)

SciPy

img = sp.imread(name, flatten=False, mode=None)

• I don't use it often, but if the above two are not possible, I will rely on it
• mode : str, optional *'L': Grayscale conversion read (8-bit) *'RGB': RGB image reading (3x8-bit) *'RGBA': 4 channel read (4x8-bit) *'I': Grayscale conversion read (32-bit signed integer) *'F': Grayscale conversion read (32-bit floating point)
• There are also'P',' CMYK', and'YCbCr'

Binalization

Basically, only Otsu is used for binarization of images, but other methods are also used depending on the purpose, so for reference.

scikit-image

#Find the threshold for each binarization method
thrOtsu = fil.threshold_otsu(img)
thrli = fil.threshold_li(img)
thrIso = fil.threshold_isodata(img)
thrYen = fil.threshold_yen(img)

#Binarization
thrImg = ((thrOtsu < img) * 255).astype(np.uint8)

OpenCV

When you do not use it at all, but you are using OpenCV consistently such as reading.

thr, thrImg = cv.threshold(img, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)

Filtering

It is mainly used for noise removal performed as a pre-processing of image processing. For noise removal, it is necessary to use various filters according to the type of noise and the characteristics of the image. Therefore, the configuration here is summarized for each type of filtering so that it can be used properly.

Gaussian Fitering Convolve with a weighted kernel that follows a Gaussian distribution. Used for general image smoothing.

#Variance of normal distribution
sigma = 1
#Kernel size
ksize = (3, 3)

# OpenCV
gauImg = cv.GaussianBlur(img, ksize=ksize, sigmaX=sigma)

# SciPy
gauImg = ndi.gaussian_filter(img, sigma=sigma)

# scikit-image
#If img is a float type-1~Must be normalized to 1
gauImg = fil.gaussian_filter(img, sigma=sigma)

Mean Filtering

Convolve with the average weight kernel. It is used to remove fine noise such as uniform noise.

#Kernel size
ksize = (5, 5)
#Weight kernel with a radius of 20 pixels
selem = disk(20)

# OpenCV
meanImg = cv.blur(img, ksize=ksize)

# scikit-image
#If img is a float type-1~Must be normalized to 1
meanImg = rank.mean(img, selem=selem)

Median Filtering

Median filtering. It is used to remove spike-like noise such as sesame salt noise.

#Kernel size
ksize = 3
#Weight kernel with a radius of 20 pixels
selem = disk(20)

# OpenCV
medImg = cv.medianBlur(img, ksize=ksize)

# scikit-learn
#If img is a float type-1~Must be normalized to 1
medImg = rank.median(img, selem=selem)

Bilateral Filtering

Since Gaussian filtering smoothes even characteristic pixels such as edges, it becomes a problem depending on the subsequent image processing. Therefore, Bilateral filtering can perform smoothing while preserving edges. The decisive difference from Gaussian filtering is not only the Gaussian distribution regarding the distance difference from the kernel center, but also the weighted kernel according to the Gaussian distribution regarding the brightness difference from the central luminance value.

#Kernel size
d = 5
#Variance of Gaussian luminance distribution
sigmaColor = 1
#Variance of Gaussian distribution
sigmaSpace = 1

# OpenCV
biImage = cv.bilateralFilter(img, d=d, sigmaColor=sigmaColor, sigmaSpace=sigmaSpace)

Morphological Operation

It mainly performs processing that greatly affects the subsequent image processing, such as noise background removal of grayscale and binary images, and feature extraction. This section also summarizes each morphology calculation process. Personally, I think it would be better to borrow only Dilation and Erosion from the library and build opening and closing functions in various ways.

Dilation

The process of expanding an object in a binary image.

#Kernel size
kernel = np.ones((3,3),np.uint8)
#Number of trials
iterations = 2

# OpenCV
dilImg = cv.dilate(thrImg, kernel=kernel, iterations=iterations)

# scikit-image
dilImg = mor.dilation(thrImg, selem=kernel)

Erosion

The process of shrinking a binary image object.

# OpenCV
eroImg = cv.erode(thrImg, kernel=kernel, iterations=iterations)

# scikit-image
eroImg = mor.erosion(thrImg, selem=kernel)

Opening

It is used when you want to remove small noise. Perform the same number of dilations after multiple erosions. At this time, the number of times of each processing depends on the magnitude of the noise to be removed.

# OpenCV
openImg = cv.morphologyEx(thrImg, cv.MORPH_OPEN, kernel=kernel, iterations=iterations)

# scikit-image
openImg = mor.opening(thrImg, selem=kernel)

Closing

It is used to fill holes that occur when an object cannot be binarized neatly due to the influence of noise and the nature of the image. Erosion the same number of times after multiple dilations. At this time, the number of processings depends on the size of the hole to be filled.

# OpenCV
closeImg = cv.morphologyEx(thrImg, cv.MORPH_CLOSE, kernel=kernel, iterations=iterations)

# scikit-image
closeImg = mor.closing(thrImg)

White Top-Hat

It is used to extract small feature patterns that existed in the background by extracting the difference between the opened image and the original image.

# OpenCV
wthImg = cv.morphologyEx(thrImg, cv.MORPH_TOPHAT, kernel=kernel, iterations=iterations)

# scikit-image
wthImg = mor.white_tophat(thrImg)

Black Top-Hat

It is used to extract small feature patterns that existed in the object by extracting the difference between the closed image and the original image.

# OpenCV
bthImg = cv.morphologyEx(thrImg, cv.MORPH_BLACKHAT, kernel=kernel, iterations=iterations)

# scikit-image
bthImg = mor.black_tophat(thrImg)

Labeling

The process of labeling each segment area in a binary image. It is mainly used for quantitative analysis after binarization (finding the area and coordinates of the center of gravity). It supports only 2D images, but I tried using it because there was a convenient function in OpenCV.

#Number of labels(nlabel)And labeling image(labImg)only
nlabel, labImg = cv.connectedComponents(img)

#Bounding box in addition to the top+area(stats)And center of gravity coordinates(centroids)Also get
nlabel, labImg, stats, centroids = cv.connectedComponentsWithStats(img)

stats: List [x, y, width, height, area] for each label centroid: Also stores the barycentric coordinates [x, y] in a list corresponding to the label

The result of actually using it for the following images is shown.

img = sk.imread('unko.png')
gimg = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
thrOtsu = fil.threshold_otsu(gimg)
thrImg = ((thrOtsu > img) * 255).astype(np.uint8)
nlabel, labelImg, stats, centroids = cv.connectedComponentsWithStats(thrImg)
colors = []
for n in range(1, nlabel):
   #Crop each poop
   tmpImg = img[stats[n][1]:stats[n][1]+stats[n][3], stats[n][0]:stats[n][0]+stats[n][2]] sk.imsave('unko' + str(n) + '.png', tmpImg)
   color.append(np.array([random.randint(0, 255), random.randint(0, 255), random.randint(0, 255)]))
leny, lenx = labelImg.shape
colorImg = np.zeros((leny, lenx, 3)).astype(np.uint8)
for x in range(lenx):
   for y in range(leny):
      for n in range(1, nlabel):
         #Colored according to the label of each poop area
         if labelImg[y][x] == n:
            colorImg[y][x] = color[n-1]
sk.imsave('unko_color.png', colorImg)

The result of cropping and saving each poop from the stats obtained from the labeling function

The result of color labeling each poop area

This is convenient! Covered with poop!

Rotation of Image

A summary of functions that rotate images. This time, I will summarize only the case of rotating from the center of the image.

#rotation angle(Counterclockwise)
angle = 90

# Scipy
rotImg = ndi.interpolation.rotate(img, angle=angle)

# OpenCV
imsize = img.shape
rotMat = cv.getRotationMatrix2D(tuple(np.array(imsize)/2), angle, scale=1.0)
rotImg = cv.warpAffine(img, rotMat, imsize)

There is also a function that flips the image.

flipCode = 0

# OpenCV
flipImg = cv.flip(img, flipCode=flipCode)

The argument flipCode is a parameter that controls the flip direction.

• 0: x-axis inversion
• Positive: y-axis inversion
• Negative: x, y both axis inversion (same as 180 degree rotation)

Interpolation

A summary of functions that perform image interpolation. There are Nearest, Bilinear, Bicubic, etc. as image interpolation methods, and they are used properly according to the image and purpose.

#Interpolation size
ipsize = (200, 200)

# OpneCV
#Interpolation method
cv.INTER_NEAREST  # Nearest
cv.INTER_LINEAR   # Bilinear
cv.INTER_CUBIC    # Bicubic
ipImg = cv.resize(img, ipsize, metLinear)

# scikit-image
#Interpolation method(Biquintic etc. are also available)
order = 0  # Nearest
order = 1  # Bilinear (default)
order = 3  # Bicubic
#preserve if img is float_range must be set to True
ipImg = tr.resize(img, ipsize, order = 3, preserve_range = True)

Extract Feature

Extract features such as edges in the image. In addition to SIFT, which is a feature extraction algorithm, edge extraction using a differential filter is also summarized in this section.

SIFT

SIFT (Scale-Invariant Feature Transform) is one of the most used feature extraction algorithms, and extracts features that are robust to scale and position changes. It is also used mainly in the field of image recognition in pattern recognition and machine learning.

The functions provided by OpenCV have been stored in the contrib module since 3.0, so they need to be added to the variant when installing. In addition, there is another method to do it in versions up to 2.4, but this time I will summarize it on the assumption that it is OpenCV 3.0 or later.

#Feature extraction
det = cv.xfeatures2d.SIFT_create()
kp = det.detect(img)   #img is CV_8U(uint8)

#Add to image
siftImg = cv.drawKeypoints(img, kp, None)

SURF

SURF (Speeded-Up Robust Features) is an algorithm that increases the processing speed instead of reducing the number of dimensions of the vector representing the feature quantity in the SIFT algorithm.

#Feature extraction
det = cv.xfeatures2d.SURF_create()
kp = det.detect(img)   #img is CV_8U(uint8)

#Add to image
surfImg = cv.drawKeypoints(img, kp, None)

Prewitt Filtering

A filtering technique for detecting the brightness gradient in an image. The concept of the Prewitt Filter is that when calculating the gradient in a certain direction in the kernel, smoothing is performed in that direction and in the horizontal direction, and then differentiation is performed.

# SciPy
preImg = ndi.filters.prewitt(img)

Sobel Filtering

It is almost the same derivative filter as the Prewitt Filter, except that it is weighted in the center of the kernel. Personally, the Sobel Filter has the impression that the edges can be removed cleanly.

# SciPy
sobImg = ndi.filters.sobel(img)

Others

Downhill Filtering

Algorithm used for background (BG) removal. See the original paper [1] for the contents.

# seed (Criteria for values considered as BG)Creation
seed = copy.deepcopy(img)
seed[1:-1, 1:-1] = img.min()

dilatedImg = mor.reconstruction(seed, img, method='dilation')
dhImg = copy.deepcopy(img - dilated)

Summary

No matter which library you use in image processing, it is dangerous to use it without understanding the algorithm, so I think it is important to consistently code with a familiar library that is familiar with the contents. Also, since my image analysis mainly targets grayscale images, I am not good at image processing in RGB images, and I have not been able to reflect it in the content of this time. In the future, I would like to learn multi-channel image processing with an eye on RGB images. The contents will be updated from time to time.

References

[1] Robinson, Kevin, and Paul F. Whelan. "Efficient morphological reconstruction: a downhill filter." Pattern Recognition Letters 25.15 (2004): 1759-1767.