[PYTHON] I tried moving food with SinGAN

Introduction

I'm sorry to disturb you in the empty space of Manufacturer / Counting Advent Calendar 2019.

Recently, I've been playing around with "SinGAN", so I'll summarize the results roughly. SinGAN is a paper that became a hot topic because it was selected as the best paper at ICCV 2019. Besides learning with just one image to generate an image of any size with the same features, it's also great.

Paper: https://arxiv.org/abs/1905.01164 Code: https://github.com/tamarott/SinGAN Video: https://youtu.be/xk8bWLZk4DU

What's more, ** I can easily try it with one image and one GPU **. It's also a nice point. With one GPU, it took several hours to learn each time in my environment.

Regarding the content of the dissertation, I have to write a master's thesis as soon as possible, so I will omit it here because there are other people who have already explained it. I found several cases in Qiita alone.

-[SinGAN] Various image generation tasks are possible from just one image -[Paper commentary] SinGAN: Learning a Generative Model from a Single Natural Image -ICCV2019 Best Paper "SinGAN" I read it briefly and summarized it briefly

By the way, in this article, I will try various functions (mainly Animation) according to SinGAN's README. If you want to reproduce it, clone the repository. stay here.

In addition, the result of playing with SinGAN is tweeted in detail at #SinGAN.

Animation To make a video from a still image, use this command in one shot.

$ python animation.py --input_name <input_file_name> 

ramen

First of all, from the classic ramen. This delicious ramen ... This street. It looks lively and delicious!

What about Jiro ramen? This chilled Chinese ... This street. It looks lively and delicious!

pizza

This pizza is ... This street. It's hot and fresh!

Bowl of rice topped with sashimi

This seafood bowl is ... This street. It's fresh!

Latte art

This latte is ... This street. It looks lively and delicious!

"Cry"

Munch's "The Scream" ... This street. The ego is collapsing!

Blue Cave (Shibuya)

Illuminations in Shibuya ... That's right. I think this was the most correct way to use it.

Harmonization To make the collage natural, you first need to train the model. Specify the background image.

$ python main_train.py --input_name <input_file_name>

The second command will harmonize the cut and pasted image and the mask image. This will be completed soon.

$ python harmonization.py --input_name <training_image_file_name> --ref_name <naively_pasted_reference_image_file_name> --harmonization_start_scale <scale to inject>

I referred to here for creating the mask image. [Create a black and white mask image of a specific area with Gimp – Urusu Lambda Web](https://urusulambda.wordpress.com/2018/07/23/gimp%E3%81%A7%E7%89%B9%E5% AE% 9A% E9% A0% 98% E5% 9F% 9F% E3% 81% AE% E7% 99% BD% E9% BB% 92% E3% 83% 9E% E3% 82% B9% E3% 82% AF% E7% 94% BB% E5% 83% 8F% E3% 82% 92% E4% BD% 9C% E6% 88% 90% E3% 81% 99% E3% 82% 8B /)

By the way, this time I would like to synthesize Santa Claus with Monet's "Water Lilies". This cut and pasted image is ... It looks like this. It's small and hard to see.

Super Resolution SinGAN can also super-resolution, so let's enlarge the previous image. This command is one shot. If nothing is specified, the height and width will be quadrupled.

$ python SR.py --input_name <LR_image_file_name>

Merry Christmas! Have a nice year!

bonus

I tried to collaborate famous paintings and Pokemon with Harmonization and SR.

Ito Jakuchu and Ho-Oh.

Turner and Lugia.

Both are not good. I still don't know the knack of Harmonization.