Introduction

I think there are many people who share photos on SNS, including Instagram. At that time, I think that the image may be processed using the application. It has various functions such as adjusting brightness and color, retouching to make the skin look beautiful, and processing photos in a sketch style.

This time, I tried simple sketch-like image processing using OpenCV.

Image processing in "sketch style" with OpenCV

environment

The environment uses Google Colaboratory. The Python version is below.

import platform
print("python " + platform.python_version())
# python 3.6.9

Image display

Now let's write the code. First, import and set the library required to display the image.

import cv2
import matplotlib.pyplot as plt
import matplotlib

%matplotlib inline
matplotlib.rcParams['image.cmap'] = 'gray'

Prepare a sample image as well. This time, we will use the free image from Pixabay.

Now, let's display the prepared sample image.

image = cv2.imread(input_file) # input_file is the path of the image

plt.figure(figsize=[10,10])
plt.axis('off')
plt.imshow(image[:,:,::-1])

Conclusion

Now let's use OpenCV to process this image like a sketch. The code is below.

def sketch(image):
    pencilSketch_img = pencilSketch(image)
    bilateral_img = cv2.bilateralFilter(image, 75, 100, 100)
    sketch_img = cv2.bitwise_and(bilateral_img, pencilSketch_img)
    return sketch_img

def pencilSketch(image):
    gray_img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    median_img = cv2.medianBlur(gray_img, 5)
    laplacian_img = cv2.Laplacian(median_img, cv2.CV_8U, ksize=5)
    _, thresh_img = cv2.threshold(laplacian_img, 100, 255, cv2.THRESH_BINARY_INV)
    pencilSketch_img = cv2.cvtColor(thresh_img, cv2.COLOR_GRAY2BGR)
    return pencilSketch_img

sketch = sketch(image)

plt.figure(figsize=[20,10])
plt.subplot(121);plt.imshow(image[:,:,::-1]);plt.axis('off')
plt.title("original image")
plt.subplot(122);plt.imshow(sketch[:,:,::-1]);plt.axis('off')
plt.title("sketch image")

The process of processing like a sketch is defined as a sketch function. The processing flow of the sketch function is as follows.

--Pencil-style image creation (pencilSketch function) --Smoothing (cv2.bilateralFilter) --Mask processing (cv2.bitwise_and)

The order is to draw a line with a pencil and paint it.

Description of each process

Each process will be explained below.

Pencil-style image creation (pencilSketch function)

The process of processing like a pencil drawing is defined as the pencilSketch function. The processing flow of the pencilSketch function is as follows.

grayscale --Smoothing (medianBlur) --Image differentiation (Laplacian)
Binarization

I will display the image.

image = cv2.imread(input_file) #Loading the original image
pencilSketch_img = pencilSketch(image) #Pencil style image

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(image[:,:,::-1])
plt.subplot(122);plt.axis('off');plt.imshow(pencilSketch_img[:,:,::-1])[

For details on the pencilSketch function, refer to here.

Smoothing (cv2.bilateralFilter)

** Smoothing **, or smoothing, is simply ** blurring an image **. Blurring an image can also be said to smooth out changes in pixel values. Noise and edges are sudden changes in pixel values. Smoothing can eliminate or obscure noise and edges.

There are several methods of smoothing, one of which is the bilateral Filter. Bilateral means "both", but this filter can leave edges nicely. The smoothing filter blurs the image, including peaks such as edges. You can use the ** bilateral filter ** to blur the image while leaving ** edges **.

For more information on smoothing, please refer to here.

The code for bilateralFilter is below.

bilateral_img = cv2.bilateralFilter(image, 75, 100, 100)

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(image[:,:,::-1])
plt.subplot(122);plt.axis('off');plt.imshow(bilateral_img[:,:,::-1])

You can see that the image is blurry.

I used cv2.bilateralFilter to smooth the image. The usage of cv2.bilateralFilter is as follows.

dst = cv2.bilateralFilter(src, d, sigmaColor, sigmaSpace[, dst[, borderType]] ) --src: Input image --dst: Output image --d: Diameter of the area of interest. --sigmaColor: Color space parameters. The larger the value, the more colors that are separated in color space affect each other. --sigmaSpace: Parameters related to coordinate space. The larger the value, the more the pixels that are separated in the coordinate space affect each other.

Mask processing (cv2.bitwise_and)

Finally, use masking to combine the color image with the pencil-style sketch image. Use cv2.bitwise_and for masking. cv2.bitwise_and is a function that performs bitwise AND processing of two images.

The masking code is below.

merged_img = cv2.bitwise_and(bilateral_img, pencilSketch_img)

plt.figure(figsize=[20,10])
plt.subplot(121);plt.axis('off');plt.imshow(bilateral_img[:,:,::-1])
plt.subplot(122);plt.axis('off');plt.imshow(merged_img[:,:,::-1])

I was able to successfully put a pencil sketch on a color image.

Summary

How was that.

This time, I tried sketch-like image processing using OpenCV. Let's review the processing flow.