[PYTHON] I tried to step through Bayesian optimization. (With examples)

Overview

Bayesian optimization is one of the methods for finding the maximum or minimum value for an unknown function f (x). Due to its nature, it is said to be suitable for the following problem cases.

--The shape of f (x) is unknown. It is not known whether it is a convex hull or whether it has bimodal characteristics. --Even if you calculate the function value f (x) once for a certain input x, the cost will increase (it will take time, etc.) --I want to find the global optimum solution, but I want to find the "maximum value (minimum value) of the estimatedf (x)" sequentially.

The next slide was easy to understand for specific algorithms and mathematical explanations.

Introduction to Bayesian Optimization for Machine Learning-Slideshare [01. Overview of Bayesian Optimization](https://colab.research.google.com/github/TatsuyaKatayama/OpenFOAMandBaysianOpt_Notebooks/blob/master/01.%E3%83%99%E3%82%A4%E3%82% BA% E6% 9C% 80% E9% 81% A9% E5% 8C% 96% E6% A6% 82% E8% AA% AC.ipynb)

Then, I found an event that I wanted to optimize a little, so I built a program for it.

Library and sample code used for Bayesian optimization

Use the GPyOpt library that was also used in the Python Notebook documentation mentioned above. This is a library that allows you to easily perform Bayesian optimization calculations in Python, and when you write sample code, it looks like this.

import GPyOpt
from numpy.core.multiarray import ndarray


def func(x: float, y: float) -> float:
    # Rosenbrock function
    #Target of this optimization. The exact solution is(x, y) = (1, 1)At the time of f(x, y) = 0
    val = 100.0 * (y - x * x) ** 2 + (1 - x) ** 2
    print(f'f({x}, {y}) = {val}')
    return val


def func_wrapper(arr: ndarray) -> float:
    #By default, all arguments are passed together as ndarray type and it is difficult to understand.
    #A function to stand and wrap with the actual function
    return func(arr[:, 0][0], arr[:, 1][0])


if __name__ == '__main__':
    #Name in each variable(Variable name)-Continuous value or discrete value-Value range.
    # type=type instead of continuous=Discrete values can also be handled
    bounds = [
        {'name': 'x', 'type': 'continuous', 'domain': (-2, 2)},
        {'name': 'y', 'type': 'continuous', 'domain': (-1, 3)},
        # {'name': 'y', 'type': 'discrete', 'domain': tuple(range(-1, 3))},
        ]

    #Problem definition. maximize=If True, aim for maximization
    problem = GPyOpt.methods.BayesianOptimization(func_wrapper, domain=bounds, maximize=False, verbosity=True)

    #Run. verbosity=When set to True, the elapsed calculation time is also displayed.
    max_iter = 25
    problem.run_optimization(max_iter, verbosity=True)

    #Graph display of the evaluation value and the transition of the best value in each enforcement in the optimization process
    problem.plot_convergence()

    #Best solution(The best evaluation value I have ever seen)Show
    print(problem.x_opt, problem.fx_opt)

    #Graph of estimated function shape(Mean / variance)And a graph showing where to look next(Acquisition function)To display
    problem.plot_acquisition()

** Evaluation value in each enforcement and transition of best value **

** A graph of the estimated function shape (mean / variance) and a graph showing where to search next (acquisition function) **

What event do you want to optimize?

** It is a white balance setting of the camera to match the color of the lens. ** **

I don't think it will be transmitted to the majority of people, so I will explain it step by step.

What is white balance?

You can get a feel for it, but even if the objects you take with the camera are the same, The color of the object changes depending on "what kind of light (lighting) it is exposed to". For example ** when it's sunny **. For example, ** in a cloudy sky **. For example, ** when you are in a room with light bulb lighting ** ... etc.

The color of the lighting is called "color temperature". In addition, correcting the photo by calculating back from the color temperature and returning it to the correct color is called "white balance adjustment".

(Quoted from https://av.jpn.support.panasonic.com/support/dsc/knowhow/knowhow31.html)

What is the color of the lens?

A lens is basically made of glass. Also, as you can see from the side of the flat glass, the glass has a slight "color". Then, the light that has passed through the glass including the lens will have a slight "color".

In addition, modern camera lenses have a thin coating. This makes the photo look even better, but this coating also has a slight "color".

(Quoted from https://av.jpn.support.panasonic.com/support/dsc/knowhow/knowhow17.html)

In addition, not all manufacturers use the same glass coating, and The type and number of lenses used will differ depending on the camera lens. Therefore, ** camera lenses have a slight difference in color **.

However, the coating, which is a technology unique to the manufacturer, has a stronger effect, and the situation is "difference between lens manufacturers> difference in glass". (Well, even the same manufacturer may have different colors depending on the brand.)

This case

Usually, Panasonic LUMIX G9 PRO and Panasonic [LEICA DG LENS](https://panasonic.jp/dc/lens/leica_dg_lens. I have attached html), but I sometimes use M.ZUIKO PRO lens. However, the two are ** slightly different in color **, so I was wondering how much to correct.

Therefore, by shooting a whitish wall with the former (* intentionally blurring the focus to make a uniform picture), and shooting the same subject ** while changing the white balance each time ** with the latter, which white Determine if the balance adjustment value is appropriate as a correction.

In the above image, there is only one dimension adjustment axis called "color temperature", but the white balance adjustment of G9 PRO is "A (orange) -B (blue) axis" and "G (green) -M (red) axis". There are two axes, and the optimum solution is searched for by adjusting these within the range of ± 9.

(Quoted from https://panasonic.jp/p-db/contents/manualdl/1428353073217.pdf)

Step through Bayesian optimization!

First, consider the function you want to optimize. ... and before that, create a function that "reads the image and calculates the color with (R, G, B)".

from typing import Tuple
from PIL import Image
from PIL.Image import BOX
from PIL.MpoImagePlugin import MpoImageFile

def get_sample_color(path: str) -> Tuple[int, int, int]:
    """Read the image file of the specified path, vertical and horizontal 1/Cut 10 spaces from the center, get the average color and return

    :param path:file name
    :return: (R, G, B)
    """
    im: MpoImageFile = Image.open(path)
    image_size = im.size
    image_size2 = (image_size[0] / 10, image_size[1] / 10)
    left_upper_pos = ((image_size[0] - image_size2[0]) / 2, (image_size[1] - image_size2[1]) / 2)
    cropped_im = im.crop((left_upper_pos[0], left_upper_pos[1],
                          left_upper_pos[0] + image_size2[0], left_upper_pos[1] + image_size2[1]))
    resized_im = cropped_im.resize((1, 1), resample=BOX)
    data_r = resized_im.getcolors()[0][1][0]
    data_g = resized_im.getcolors()[0][1][1]
    data_b = resized_im.getcolors()[0][1][2]
    return data_r, data_g, data_b

You also need a function to calculate the difference in color. Convert to YCbCr color space instead of RGB space and calculate the L2 norm for Cb and Cr.

def calc_color_diff(color1: Tuple[int, int, int], color2: Tuple[int, int, int]) -> float:
    """Calculate the difference between two colors

    :param color1:Color 1
    :param color2:Color 2
    :return:Difference between two colors in YCrCr space
    """
    #JPEG YCbCr diff
    cb_1 = -0.1687 * color1[0] - 0.3313 * color1[1] + 0.5 * color1[2] + 128
    cr_1 = 0.5 * color1[0] - 0.4187 * color1[1] - 0.0813 * color1[2] + 128
    cb_2 = -0.1687 * color2[0] - 0.3313 * color2[1] + 0.5 * color2[2] + 128
    cr_2 = 0.5 * color2[0] - 0.4187 * color2[1] - 0.0813 * color2[2] + 128
    return (cb_1 - cb_2) ** 2 + (cr_1 - cr_2) ** 2

And the function I want to optimize, but unlike the general Pyhon function, It is necessary to perform the process of "** Stop the process until it can be read by taking a picture **". …… I thought that the quickest way was the ʻinput ()` function, so I implemented it using that.

import os

def get_score(a: float, g: float) -> float:
    """Score in the specified settings(= Distance from the standard color)To calculate

    :param a:Parameter A(amber)The value of the. B if it is a negative number(blue)Become the color of the side
    :param g:Parameter G(green)The value of the. Negative number is M(Magenta)Become the color of the side
    :return:The distance between shades. The smaller the better
    """

    #Generate the name of the file you want to read
    if a >= 0.0:
        temp1 = f'A{int(a)}'
    else:
        temp1 = f'B{int(-a)}'
    if g >= 0.0:
        temp2 = f'G{int(g)}'
    else:
        temp2 = f'M{int(-g)}'
    file_name = f'MZD_{temp1}{temp2}.jpg'

    #Check if the file does not exist and create it if it does not exist
    while not os.path.exists(file_name):
        _ = input(f'{file_name}After creating, press Enter.')

    #Read the file and average color near the center(= Measured value)To get
    mzd_rgb = get_sample_color(file_name)

    #Compare with the color of the sample data
    score = calc_color_diff(mzd_rgb, base_rgb)
    print(f'a={a} g={g} score={score}')
    return score

The rest of the flow is almost the same. Please note that this input (white balance setting value) is a discrete value, so it is necessary to set 'type':'discrete' for the discrete value. Well, as mentioned above, even with 'type':'discrete', an integer value (ʻint type) is not passed to the function you want to optimize, but it is always a float` type.

from numpy.core.multiarray import ndarray

def get_score2(x: ndarray) -> float:
    """Score in the specified settings(= Distance from the standard color)To calculate"""
    return get_score(x[:, 0][0], x[:, 1][0])


if __name__ == '__main__':
    bounds = [{'name': 'a', 'type': 'discrete', 'domain': tuple(range(-9, 10))},
              {'name': 'g', 'type': 'discrete', 'domain': tuple(range(-9, 10))}]

    myProblem = GPyOpt.methods.BayesianOptimization(get_score2,
                                                    domain=bounds,
                                                    maximize=False,
                                                    verbosity=True)
    max_iter = 25
    myProblem.run_optimization(max_iter, verbosity=True)
    myProblem.plot_convergence()
    print(myProblem.x_opt, myProblem.fx_opt)
    myProblem.plot_acquisition()

Optimization result

It looks like this. Since the evaluation value was a little sticky, the convergence was poor, but the results were generally satisfactory.

If you want to modify it further

"Color" is not only about white balance, but also about the contrast of the picture. For example, some people feel that the contrast is different depending on whether it is the LUMIX brand or the LEICA brand.

This time, I adjusted it with "white blurred wall (≒ 18% gray)", so I can only adjust the white balance, but By adjusting with "Object with stepwise white to black color printed (gray chart)" The degree of neutral color ... In other words, the contrast can also be tuned.

Not only Bayesian optimization but also 3D or more input is supported, so If you want to make finer adjustments, try modifying the above program.