[PYTHON] Proposal of Kuwahara filter as a photographic expression

Introduction

** What is Kuwahara Filter ** Please refer to the above for an explanation of the Kuwahara filter.

This time

This time, I made a quick suggestion at the end of the above.

"It might be interesting to have a depth map and adjust the size of the square area accordingly ..."

I would like to actually try.

This idea is that if the main subject is not clear (out of focus anywhere), it looks more like a background than a painting, compared to the previous image with a constant square area and a filter. It was obtained based on.

Implementation

In response to the previous article, there was an opinion that the meaning of the code was not understandable (difficult), so I wrote it with the "Sonomanma" code, which is out of the calculation speed. However, leaving too many loops in Python would be very time consuming, so I helped the JIT compiler. If you don't have Numba (a library that includes JIT), How to use the JIT compiler library Numba to dramatically speed up Python code Please refer to 02/155433).

Kuwahara_with_Depth.py

import matplotlib.pyplot as plt
import numpy as np
import cv2
from numba import jit


@jit
def mean(arr):
    h, w, c = arr.shape
    avg = np.array([0]*c)
    for i in range(h):
        for j in range(w):
            for k in range(c):
                avg[k] += arr[i, j, k]
    return avg/(h*w)


@jit
def var(arr, mean):
    h, w, c = arr.shape
    vari = 0
    for i in range(h):
        for j in range(w):
            for k in range(c):
                vari += (arr[i, j, k]-mean[k])**2
    return vari


@jit
def kuwahara_with_depth(pic, r, r_min, depth):
    h, w, c = pic.shape
    out = np.empty_like(pic)
    pic = np.pad(pic, ((r, r), (r, r), (0, 0)), "edge")
    depth = depth/depth.max()
    surr = ((1, 0), (0, 1), (1, 1))
    for i in range(h):
        for j in range(w):
            dr = max(r_min, int(depth[i, j]*r))
            arr = pic[i+r-dr:i+r, j+r-dr:j+r]
            avg = mean(arr)
            var_min = var(arr, avg)
            color = avg
            for s, t in surr:
                arr = pic[i+r-(1-s)*dr:i+r+s*dr, j+r-(1-t)*dr:j+r+t*dr]
                avg = mean(arr)
                vari = var(arr, avg)
                if vari < var_min:
                    color = avg
                    var_min = vari
            out[i, j] = color
    return out


def main(picpath, r, r_min, rate, depthpath): #Input image path, maximum value on one side of the square area, minimum value on one side of the square area, image size reduction ratio, depth map path
    pic = np.array(plt.imread(picpath))
    pic = cv2.resize(pic, (int(pic.shape[1]*rate), int(pic.shape[0]*rate)))
    depth = cv2.resize(np.array(plt.imread(depthpath)[:, :, 0]), (pic.shape[1], pic.shape[0]))
    # depth=cv2.resize(np.rot90(np.array(plt.imread(depthpath))[:,:,0]),(pic.shape[1],pic.shape[0])) #For when the orientation of the depth map does not match the original image
    fpic = kuwahara_with_depth(pic, r, r_min, depth).astype(pic.dtype)
    plt.imshow(fpic)
    # plt.imshow(np.rot90(fpic,3)) #For when the output image is facing sideways
    plt.show()


picpath = "input_pic.jpg " #Input image path
depthpath = "depthmap.jpg " #Depth map image path

if __name__ == "__main__":
    main(picpath, 20, 3, 0.5, depthpath)

result

Again, I will use ** French Nekko ** as a sample from the French photograph. In the above explanation, it was expressed as a depth map, but of course there is no such data, so I will create an image that looks like it in some way. This time I made it properly with Photoshop. By the way, the whiter the area, the larger one side of the square area. (It seems that the photo file of the smartphone that can use some background blur contains the information of the depth map. Stereo image / multi-view image creation using Google Camera Deph map .org / kitkat / index.html)))

Here is the one that has been filtered this time using these. Please click to enlarge and see. It's a mysterious finish, like a natural fusion of a photo and a picture, like blurring with paint instead of blurring the lens.

Summary

Even in the photograph used in the sample, the impression will change greatly depending on the maximum and minimum values of the square area. If adjusted properly, it may look like a painting in which the subject and background are clearly drawn, and in these days when photogenicity is required, I think it can be used as an unusual photographic expression as shown in the title. Why don't you give it a try?