[PYTHON] Let's talk about the tone curve of image processing ~ LUT is amazing ~

Introduction

I've just started using ʻOpneCV` in Python I tried various things and stumbled I'm going to write it hard.

When you pick it up and talk ** LUT is amazing! ** I thought ** I tried using LUT! ** It is the content.

(As a result) I also compared the performance with my own color tone conversion function.

Operating environment

Terminal: Windows 10
Console: cmd(command prompt)
python：3.6.8
Virtual environment: venv

If you want to make cmd comfortable, click here >> [https://qiita.com/RoaaaA/items/694ae4ccfa4a69fc8286)

Introduction

I learned about Computer Vision (CV) in a university class (paperwork). For the first time, I came across something called "** tone curve **". What is a tone curve [^ 1]

Each pixel of a digital image has a value (pixel value) that represents its shading. In order to change the shading of an image, how is the pixel value of the output image associated with the pixel value of the input image? The function that gives such a correspondence is called the gray-level transformarion function, and the graphic representation of it is called the tone curve.

Looking at the net, it seems to be familiar to those who like cameras.

Among the tone curves that appeared in the textbook, there were the following two. (* Of course, many others have appeared)

The horizontal axis is ** input pixel value **, and the vertical axis is ** output (converted) pixel value **.

Figure 1	Figure 2

Put simply ** Contrast conversion ** and ** Conversion that lowers contrast **.

This part of the class was already over half a year ago. I wanted to actually apply this tone curve to the image.

Functionalization

Expressed as a function, it is as follows (x is the input pixel value 0 to 255).

Function in Figure 1:

f(x) = \begin{cases} 2 \cdot x & (x < 128) \\ 255 & (otherwise)
  \end{cases}

Function in Figure 2:

f(x) = \begin{cases} 0 & (x < 128) \\ 2 \cdot x - 255 & (otherwise)
  \end{cases}

It looks like it can be generalized, but let's leave that for now.

Failure 1

First of all, I decided to write the code shown in Figure 1. But ... it got stuck immediately.

When I came up with the above function I thought I could simply double the element values in the original array of pixel values.

Conversion function

def toneCurve(frame):
    return 2 * frame

frame = cv2.imread('.\\image\\castle.jpg')
cv2.imwrite('cas.jpg', toneCurve(frame))

frame is an array of pixel values

But it didn't work. The output image is as follows.

The original image	Converted image

Apparently, if you simply double the pixel value obtained by cv2.imread () Anything greater than 255 (maximum) will be 255 instead of being complemented It seemed to convert the value cyclically and return it. (ex:256=0, 260=4)

It ’s very horrible and I like this one The purpose is not this image, so I will improve it.

Improvement 1 (Failure 2)

If you look it up, you can use cv2.add () It seems that values above 255 are treated as 255.

I tried.

Improvements in Figure 1

Add your own pixel value to yourself = constant times Because it means The conversion function looks like this:

Conversion function

def toneCurve11(frame, n = 1):
    tmp = frame
    if n > 1:
        for i in range(1,n):
            tmp = cv2.add(tmp, frame)
    return tmp

frame is an array of pixel values n represents the number of times the pixel value is multiplied, that is, the number of times it is added.

Then it is the result.

The original image	Converted image

Yes! This is what I was looking for! It output an image with increased contrast. (Was the grayscale image better?)

However, how can we achieve the conversion shown in Fig. 2 by just adding this method? Do you want to output an image?

** I came up with a quick hit **: fireworks:

Improvements in Figure 2

What I came up with is this.

** Negative / Positive Inversion> cv2.add ()> Negative / Positive Inversion **

It looks like this in the figure.

1.No conversion	2.Negative / positive reversal
3.cv2.add()Doubled with	4.Negative / positive reversal

The changes are in the order of 1, 2, 3, 4.

The conversion code is as follows.

Conversion function

def negaPosi(frame):
    return 255 - frame


def toneCurve21(frame, n = 1):
    if n > 1:
        return negaPoso(toneCurve11(negaPosi(frame), n))
    return frame

As before frame is an array of pixel values n represents the number of times the pixel value is multiplied, that is, the number of times it is added.

Well then, the result.

The original image	Converted image

The image with reduced contrast was output safely.

Pause

Even if I do the above, it works well. I felt like I was being asked about ** order **, so I decided to look into other methods.

Then, cv2 seems to have a convenient one called ** LUT (Look Up Table) **. Moreover, if the order can be made quite small ...

LUT I've taken some approaches above, but in the end If you replace the acquired pixel value with another pixel value, you can convert it.

Here, the possible values of pixel values are ** at most 256 ways ** from 0 to 255. If so, have a correspondence table in which the conversion destination value for a certain pixel value is registered in advance. When referring to a pixel value, instead of calculating the pixel value itself Computational costs can be reduced by simply replacing them.

Here is a brief description of the LUT.

For example, the following correspondence table can be considered.

Yes, this table is the correspondence table of the tone curves in Figure 1.

If the pixel value of frame [90] [90] is If it was [12, 245, 98] After applying the LUT It becomes [24, 255, 196].

Improvement 2

We will actually use the LUT.

Improvement of Figure 1 (Re :)

Conversion function

def toneCurve12(frame, n = 1):
    look_up_table = np.zeros((256,1), dtype = 'uint8')
    for i in range(256):
        if i < 256 / n:
            look_up_table[i][0] = i * n
        else:
            look_up_table[i][0] = 255
    return cv2.LUT(frame, look_up_table)

frame is an array of pixel values n indicates how many times the pixel value should be multiplied.

Function description

    look_up_table = np.zeros((256,1), dtype = 'uint8')

For the part of, create a correspondence table (256 x 1) for conversion.

At this stage, the following values are still stored.

>>> look_up_table = np.zeros((256,1), dtype = 'uint8')
>>> look_up_table
array([[0],
       [0],
       [0],
       ...,
       [0],
       [0]], dtype=uint8)
>>> len(look_up_table)  
256

From the next line, we will store the values referenced in this array.

    for i in range(256):
        if i < 256 / n:
            look_up_table[i][0] = i * n
        else:
            look_up_table[i][0] = 255

Regarding the part of, we will register the pixel values in the correspondence table for conversion created earlier.

Also

        if i < 256 / n:

The part of is a generalization of the conditional expression so that it works even if the pixel value is n times. (ex: n = 3 [3 times the pixel value])

    return cv2.LUT(frame, look_up_table)

Regarding the part of, the correspondence is taken by the LUT registered earlier, Returns the converted image data.

Well then, the result.

The original image	Improvement 1	Improvement 2

You can get the desired image.

Let's take a look at the execution time (ms).

Processing is faster.

In the same way, we will improve Figure 2.

Improvement of Figure 2 (Re :)

Conversion function

def toneCurve22(frame, n = 1):
    look_up_table = np.zeros((256,1), dtype = 'uint8')
    for i in range(256):
        if i < 256 - 256 / n :
            look_up_table[i][0] = 0
        else:
            look_up_table[i][0] = i * n - 255 * (n - 1)
    return cv2.LUT(frame, look_up_table)

What I'm doing is almost the same, so I won't explain it in detail,

        if i < 256 - 256 / n :

The part of is a generalization of the function part when n = 2 where x <128. If n = 2 x <256 --256 / n = 128 That is, It can be transformed into x <128.

Well then, the result.

The original image	Improvement 1	Improvement 2

You can get the desired image.

Let's take a look at the execution time (ms).

After all, the one that processes quickly is good.

in conclusion

This time, I talked about the tone curve lazily.

OpenCV is very convenient. My ancestors don't get upset ... What's more, Python can be used with just one command.

Will anyone see this article? Or, even though I'm worried If this is a memorandum, insure I always post.

I hope this article will be useful to someone someday.

Finally, put the source code used this time and finish. What should I do next?

Then: wave:

Source code

Under the current directory In a directory called image There is an image file called castle.jpg. If you use this code, please change it yourself.

Also, install the library below.

console

$ pip install opencv-python

$ pip freeze
numpy==1.18.4
opencv-python==4.2.0.34

ex.py

import cv2
import numpy as np


def toneCurve(frame):
    return 2 * frame


def negaPosi(frame):
    return 255 - frame


def toneCurve11(frame, n = 1):
    tmp = frame
    if n > 1:
        for i in range(1, n):
            tmp = cv2.add(tmp, frame)
    return tmp


def toneCurve12(frame, n = 1):
    look_up_table = np.zeros((256, 1), dtype = 'uint8')
    for i in range(256):
        if i < 256 / n:
            look_up_table[i][0] = i * n
        else:
            look_up_table[i][0] = 255
    return cv2.LUT(frame, look_up_table)


def toneCurve21(frame, n = 1):
    if n > 1:
        return negaPoso(toneCurve11(negaPosi(frame), n))
    return frame


def toneCurve22(frame, n = 1):
    look_up_table = np.zeros((256, 1), dtype = 'uint8')
    for i in range(256):
        if i < 256 - 256 / n :
            look_up_table[i][0] = 0
        else:
            look_up_table[i][0] = i * n - 255 * (n - 1)
    return cv2.LUT(frame, look_up_table)


def main():
    img_path = '.\\image\\castle.jpg'
    img = cv2.imread(img_path)
    cv2.imwrite('.\\image\\tone00.jpg', toneCurve(img))
    cv2.imwrite('.\\image\\tone11.jpg', toneCurve11(img, 2))
    cv2.imwrite('.\\image\\tone12.jpg', toneCurve12(img, 2))
    cv2.imwrite('.\\image\\tone21.jpg', toneCurve21(img, 2))
    cv2.imwrite('.\\image\\tone22.jpg', toneCurve22(img, 2))

if __name__ == '__main__':
    main()

reference

-[Computer Graphics] cv -Optie Lab (Hatena Blog) -Python learning process (Hatena Blog) -[Image source](https://pixabay.com/ja/photos/%E5%9F%8E-%E3%82%B9%E3%82%B3%E3%83%83%E3%83%88 % E3% 83% A9% E3% 83% B3% E3% 83% 89-3619698 /) (Pixabay)

[^ 1]: Excerpt from [Computer Graphics] cv