Introduction

I think there are many people who share photos on SNS, including Instagram. At that time, I think that the image may be processed using the application. It has various functions such as adjusting brightness and color, retouching to make the skin look beautiful, and processing photos in a sketch style.

This time, I tried a simple pencil drawing style image processing using OpenCV.

Image processing in "pencil style" with OpenCV

environment

The environment uses Google Colaboratory. The Python version is below.

import platform
print("python " + platform.python_version())
# python 3.6.9

Image display

Now let's write the code. First, import and set the library required to display the image.

import cv2
import matplotlib.pyplot as plt
import matplotlib

%matplotlib inline
matplotlib.rcParams['image.cmap'] = 'gray'

Prepare a sample image as well. This time, we will use the free image from Pixabay.

Now, let's display the prepared sample image.

image = cv2.imread(input_file) # input_file is the path of the image

plt.figure(figsize=[10,10])
plt.axis('off')
plt.imshow(image[:,:,::-1])

Conclusion

Now let's use OpenCV to process this image like a pencil drawing. I tried to display it together with the original image.

def pencilSketch(image):
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    median = cv2.medianBlur(gray, 5)
    laplacian = cv2.Laplacian(median, cv2.CV_8U, ksize=5)
    _, thresh = cv2.threshold(laplacian, 100, 255, cv2.THRESH_BINARY_INV)
    pencilSketchImage = cv2.cvtColor(thresh, cv2.COLOR_GRAY2BGR)
    return pencilSketchImage

pencilSketchImage = pencilSketch(image)

plt.figure(figsize=[20,10])
plt.subplot(121);plt.imshow(image[:,:,::-1]);plt.axis('off')
plt.title("original image")
plt.subplot(122);plt.imshow(pencilSketchImage[:,:,::-1]);plt.axis('off')
plt.title("pencil sketch image")

The process of processing like a pencil drawing is defined as the pencilSketch function. The processing flow of the pencilSketch function is as follows.

grayscale --Smoothing (medianBlur) --Image differentiation (Laplacian)
Binarization

Description of each process

Each process will be explained below.

grayscale

** Grayscale ** (Grayscale or grayscale) is a type of color representation. Simply put, turning a color image into a black and white image.

The grayscale code is below.

image = cv2.imread(input_file) #Loading the original image
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) #grayscale

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(image[:,:,::-1])
plt.subplot(122);plt.axis('off');plt.imshow(gray)

I'm using cv2.cvtColor to convert from color to grayscale. The usage of cv2.cvtColor is as follows.

dst = cv2.cvtColor(src, code) --src: Input image --dst: Output image --code: Color space conversion method

Smoothing (medianBlur)

** Smoothing **, or smoothing, is simply ** blurring an image **. Blurring an image can also be said to smooth out changes in pixel values. Noise and edges are sudden changes in pixel values. Smoothing can eliminate or make edges less noticeable.

There are several methods for smoothing, and median Blur is one of them. median is the "median", which takes the median from the pixels contained in the specified kernel and fills the entire kernel with that value.

The "kernel" is the area of the image of interest, but if it is difficult, you can use the image of a "box".

For more information on smoothing, please refer to here.

The code for medianBlur is below.

median = cv2.medianBlur(gray, 5) # medianBlur

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(gray)
plt.subplot(122);plt.axis('off');plt.imshow(median)

You can see that the image is blurry.

I used cv2.medianBlur to blur a grayscale image. The usage of cv2.medianBlur is as follows.

dst = cv2.medianBlur(src, ksize) --src: Input image --dst: Output image --ksize: Kernel size (however, odd number of 3 or more)

Image differentiation (Laplacian)

** Derivative of image ** detects the part where the pixel value changes rapidly, that is, ** edge **.

There are several ways to differentiate an image, the Laplacian is one of them. The Laplacian filter is a double derivative and is effective when you want finer edge detection than the Sobel filter with a single derivative.

For more information on image differentiation, see here.

The Laplacian code is below.

laplacian = cv2.Laplacian(median, cv2.CV_8U, ksize=5) # Laplacian

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(median)
plt.subplot(122);plt.axis('off');plt.imshow(laplacian)

You can see that the edges have been detected.

I used cv2.Laplacian to differentiate the image. The usage of cv2.Laplacian is as follows.

dst = cv2.Laplacian(src, ddepth, ksize) --src: Input image --dst: Output image --ddepth: Output image type (bit depth) --ksize: Kernel size (odd number)

Binarization

Binarization is the process of converting an image into two values (binary), white and black. It is different from the gray scale that displays between white and black in stages. A value called a threshold (threshold) is determined, and if the pixel value is larger than that, it is converted to white, and if it is smaller, it is converted to black.

There are several ways to binarize an image. This time, we performed general binarization.

For more information on binarization, please refer to here.

The binarization code is below.

_, thresh = cv2.threshold(laplacian, 100, 255, cv2.THRESH_BINARY_INV) #Binarization

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(laplacian)
plt.subplot(122);plt.axis('off');plt.imshow(thresh)

Only those with relatively high pixel values could be left.

I used cv2.threshold to binarize the image. The usage of cv2.threshold is as follows.

ret, dst = cv2.threshold(src, thresh, maxVal, method) --src: Input image (grayscale) --ret: Threshold value used --dst: Output image --thressh: threshold --maxVal: Maximum value (value above the threshold) --method: Threshold processing - cv2.THRESH_BINARY --cv2.THRESH_BINARY_INV etc.

This time, the threshold is set to 100. This time, the threshold value (100) of the return value is not used, so it is set to _ (underscore).

In the threshold processing, cv2.THRESH_BINARY_INV is used to invert black and white. cv2.THRESH_BINARY_INV is the process to set the pixel that exceeds the threshold (100 this time) to 0 (black) and the other pixels to maxVal (255 this time).

Summary

How was that.

This time, I tried to sketch like a pencil drawing using OpenCV. Let's review the processing flow.

grayscale --Smoothing (medianBlur) --Image differentiation (Laplacian)
Binarization

By changing parameters such as threshold value and kernel size, you can change the fineness of lines. I think it would be interesting to output an image by changing various parameters.

[PYTHON] I tried to process the image in "pencil style" with OpenCV