I tried "smoothing" the image with Python + OpenCV
Introduction
Image processing does not always provide beautiful images. It is often unclear or noisy. If there is noise, it may be removed using a technique called smoothing. Alternatively, you may intentionally create a blurry image and use it as dummy data.
This time, we will use Python to smooth the image with OpenCV.
What is smoothing?
Smoothing, or smoothing, is simply ** blurring an image **. Blurring an image can also be said to smooth out changes in pixel values. Noise and edges are sudden changes in pixel values. Smoothing can eliminate or obscure noise and edges.
Preparation
The environment uses Google Colaboratory. The Python version is below.
import platform
print("python " + platform.python_version())
# python 3.6.9
Let's display the image
Now let's write the code. First, import OpenCV.
import cv2
In addition, import the following to display the image in Colaboratory.
from google.colab.patches import cv2_imshow
Prepare a sample image as well. This time, we will use the free image from Pixabay.
Now, let's display the prepared sample image.
img = cv2.imread(path) #path specifies where the image is placed
cv2_imshow(img)
Also, prepare an image with noise so that you will need it later. Here, let's add a noise called "salt-and-pepper noise (also called salt-and-pepper noise)" to the image.
# salt-and-pepper noise
#See below for code
# https://lp-tech.net/articles/nCvfb?page=2
import numpy as np
row, col, ch = img.shape
img_sp = cv2.imread(path)
# salt
pts_x = np.random.randint(0, col-1 , 1000)
pts_y = np.random.randint(0, row-1 , 1000)
img_sp[(pts_y,pts_x)] = (255, 255, 255)
# pepper
pts_x = np.random.randint(0, col-1 , 1000)
pts_y = np.random.randint(0, row-1 , 1000)
img_sp[(pts_y, pts_x)] = (0, 0, 0)
cv2_imshow(img_sp)
Smoothing
General smoothing
Smoothing is simply blurring an image. The easiest way to smooth an image with OpenCV is to use cv2.blur. Here, blur means "blurring".
Now let's display the smoothed image. I will display it side by side with the original image.
img_blur = cv2.blur(img, (3, 3))
imgs = cv2.hconcat([img, img_blur])
cv2_imshow(imgs)
The left is the original image and the right is the smoothed image. You can see that the right one is a little out of focus.
There are two arguments for cv2.blur. The first is the input image. The second is called the kernel. When one point of the image is decided, it shows how much area around it is included. Think of it as the size of a box.
In the above example, it is (3, 3), which means that the target is a 3x3 area centered on one point in the image. In cv2.blur, which is a general smoothing, the kernel is filled with the average value of pixels in this kernel. The larger the kernel size, the stronger the blurring of the image.
Let's display an image with various kernel sizes changed.
img1 = cv2.blur(img, (1, 1))
img2 = cv2.blur(img, (2, 2))
img3 = cv2.blur(img, (3, 3))
img4 = cv2.blur(img, (4, 4))
img5 = cv2.blur(img, (5, 5))
img6 = cv2.blur(img, (6, 6))
imgs_1 = cv2.hconcat([img1, img2, img3])
imgs_2 = cv2.hconcat([img4, img5, img6])
imgs = cv2.vconcat([imgs_1, imgs_2])
cv2_imshow(imgs)
From the upper left, the image shows the kernel size increased. You can see that the blurring of the image is getting stronger and stronger.
Gaussian filter
In addition to general smoothing, OpenCV allows for some smoothing processes.
Next, I will introduce a Gaussian filter. In general smoothing, the pixels in the kernel are filled with a fixed value called the average value. For Gaussian filters, the value changes depending on the distance from the center of the kernel. The value is highest at the center and decreases as the distance increases. It is called a Gaussian filter because it follows a function called a Gaussian function. The formula for the Gaussian function is:
\frac{1}{\sqrt{2\pi\sigma^2}}\exp\Bigl(-\frac{x^2}{2\sigma^2}\Bigr) \\
The graph of the Gaussian function is as follows.
Now let's smooth the image with a Gaussian filter. I will also display this side by side with the original image.
img_gauss = cv2.GaussianBlur(img, (3, 3), 3)
imgs = cv2.hconcat([img, img_gauss])
cv2_imshow(imgs)
cv2.GaussianBlur has three arguments. The first two, like cv2.blur, are the input image and kernel size. The third argument corresponds to the Gaussian function $ \ sigma $. If $ \ sigma $ is small, the peak will be higher but the spread will be narrower. Conversely, the larger $ \ sigma $, the larger the spread but the lower the peak.
median filter
Next, I will introduce the median filter. median is the "median", which takes the median from the pixels contained in the specified kernel and fills the entire kernel with that value. The difference from general smoothing is that it always uses existing pixel values, not averaged pixel values.
Let's compare the image using the median filter with the original image.
img_med = cv2.medianBlur(img, 3)
imgs = cv2.hconcat([img, img_med])
cv2_imshow(imgs)
cv2.medianBlur has two arguments, the input image and the kernel size. In the above, the second argument is 3, which represents a 3x3 kernel.
This median filter is very useful for removing salt-and-pepper noise. Actually, the result of applying the median filter to the image with noise is as follows.
You can see that it has been removed cleanly.
bilateral filter
Finally, I would like to introduce a bilateral filter.
Bilateral means "both are", but this filter can leave edges nicely. Until now, smoothing filters have blurred peaks such as edges. The bilateral filter is a convenient filter that allows you to blur an image while leaving edges. Let's output it together with the original image.
img_bi = cv2.bilateralFilter(img, 9, 75, 75)
imgs = cv2.hconcat([img, img_bi])
cv2_imshow(imgs)
You can see that the edges are left firmly even though it is smooth overall.
I will omit the details of the argument of cv2.bilateralFilter. Please refer to the official OpenCV documentation.
Summary
This time, I used Python to smooth the image with OpenCV.
If you need to remove noise in your image processing, try smoothing.
For more details on smoothing, please refer to the following.
Recommended Posts