Exploring image filter parameters with blackbox optimization in Python Note

background

If the target function (problem) is like a black box, it is necessary to find its parameters.

It seems that it is called hyperparameter optimization (search) in the field of blackbox optimization and machine learning.

There are often requests for blackbox optimization in image processing as well.

For example, you want to match the appearance of the target oceanic image (instagram image) to make it an oceanic image even if you take a slightly unsightly photo. In this case, brightness or sepia/film filter. Search for filter parameters such as. (If you find it manually, it will melt infinitely and it will be hard)

This time, I will simplify the problem a little more and estimate the parameters of the blur with the image blurred by ImageMagick.

I will use RMSE for the error of the image for the time being (The more they match, the lower the value)

A note on optimizing blackbox functions in Python https://qiita.com/syoyo/items/6c33acb0fd475e651f2f

A note that runs an external program in Python and parses the resulting line https://qiita.com/syoyo/items/d13af423604192cee41c

Let's move ImageMagick with an external command and optimize it with benderopt, referring to.

Data and code

Before blurring

After blurring (-blur 27x20)

from benderopt import minimize
import numpy as np
import logging
import subprocess
import parse

logging.basicConfig(level=logging.DEBUG) # logging.INFO will print less information

blurred_filename = "shimokita-blur.jpg " # convert -blur 27x20
ref_filename = "shimokita.jpeg "

k_num_eval = 10000

def extract_result(lines):
    for line in lines:
        print(line.decode("utf-8"))
        ret = parse.parse("{:g} ({:g})", line.decode("utf-8"))
        if ret:
            return ret[0]

    raise RuntimeError("Failed to extract value from result.")

count = 0

def f(radius, sigma):
    global count

    count += 1

    print("run {} of {} ...".format(count, k_num_eval))
    tmp_filename = "shimokita-tmp.jpg "
    cmd = "convert {} -blur {}x{} {}".format(ref_filename, radius, sigma, tmp_filename)
    ret = subprocess.run(cmd, shell=True)


    # Compare two images using RMSE
    cmp_cmd = "compare -metric rmse {} {} null:".format(tmp_filename, blurred_filename)
    ret = subprocess.run(cmp_cmd, shell=True, capture_output=True)
    # `compare`(ImageMagick) outputs result into stderr, not stdout
    lines = ret.stderr.splitlines()

    val = extract_result(lines)

    return val


# We define the parameters we want to optimize:
optimization_problem_parameters = [
    {
        "name": "radius",
        "category": "uniform",
        "search_space": {
            "low": 0,
            "high": 100,
        }
    },
    {
        "name": "sigma",
        "category": "uniform",
        "search_space": {
            "low": 0,
            "high": 100,
        }
    }
]

# We launch the optimization
best_sample = minimize(f, optimization_problem_parameters, number_of_evaluation=k_num_eval)

print("radius", best_sample["radius"])
print("sigma", best_sample["sigma"])
print("err = ", f(best_sample["radius"], best_sample["sigma"]))

result

I turned it 10,000 times.

radius 27.993241302558445
sigma 19.957452459359814

err =  49.5512

Voila!

The radius is 1 different from 27 (true value) and 28 (estimated value), but the result is close to the true value.

Let's take the difference with idiff.

https://openimageio.readthedocs.io/en/release-2.2.8.0/idiff.html

(The difference is 20 times. If the difference is the same size, it is visually almost black (= match))

jpg The effect of compression seems to be great ...

By the way, 100 times was not good at all, and 1000 times was reasonably close.

TODO

• [] I want to try Bayesian Optimization.
• [] For images, use uncompressed format (PNG, BMP, TIFF).
• [] If you write the result in a file, you will be worried about disk I/O consumption (SSD seems to shorten the life?), So consider whether you can use memdisk.
• [] benderopt does not support parallel processing, so if you execute a non-multithreaded external program, it will take a long time to process, so consider using a library that can be parallelized.
• [] Operate around Photoshop in the same way to master the generation of Instagram-worthy images (like command line control)