[PYTHON] [Introduction to StyleGAN] I played with "A woman transforms into Mayuyu" ♬

You can now learn the Style vector of the mapping latent space of your own image. So, I played with "A woman transforms into Mayuyu". This time, it was necessary to install dlib for self-image learning, and I was able to install it safely by following the reference ①. As a result, the Style vector $ w $ of Mayuyu's latent space could be obtained, and the following transformation video was created.

【reference】 ① When you get stuck with dlib installation on windows

What i did

・ Installation of dlib ・ Pre-processing with align_images.py ・ Mayuyu and mixing ・ Try "a woman transforms into Mayuyu"

・ Installation of dlib

StyleGAN requires image pre-processing before learning. The code in align_images.py has the following description.

Extracts and aligns all faces from images using DLib 
and a function from original FFHQ dataset preparation step
    python align_images.py /raw_images /aligned_images

This means that all face images need to be pre-processed in align_images.py. And it was a difficult place to install this Dlib. However, as shown in Reference ① above, it worked fine with the following procedure.

• pip install cmake --Install Visual Studio 2019 Community (check desktop development with C ++)
• pip install dlib

・ Pre-processing with align_images.py

All you have to do is put the raw data in / raw_imges and run the following command:

python align_images.py /raw_images /aligned_images

As a result, the following aligned_image is output.

	raw_image	aligned_image
size	250x235	1024x1024
image

At this stage, the face part of the image is cut out and enlarged to 1024x1024. Using this image, the Style vector mayuyu.npy of the mapping latent space is learned by the following command and stored in latent /.

python encode_images.py aligned_images/ generated_images/ latent/

・ Mayuyu and mixing

Using the Style vector of this latent space, you can mix with various images as you did last time. See below for the code. StyleGAN/mayuyu_mix.py However, the changes to $ w $ in StyleGAN are as follows.

style_ranges=[range(1,18)]+[range(2,18)]+[range(3,18)]+[range(4,18)]+[range(5,18)]+[range(6,18)]

The results are as follows. It is interesting that a pretty beautiful woman appears by arranging with a man. As a whole, men below range (4,18) are becoming feminine, and other people are beginning to have more Mayuyu-side characteristics than the original. In other words, range (0,3) takes over a considerable amount of global features in Mayuyu.

・ Try "a woman transforms into Mayuyu"

So, at the end, I will try mixing instead of exchanging. The main parts of the code are: The whole is placed below. StyleGAN/simple_dlatents_mixing2.py

simple_dlatents_mixing2.py

    # Pick latent vector.
    rnd = np.random.RandomState(6) #5
    latents1 = rnd.randn(1, Gs.input_shape[1])
    print(latents1.shape)
    # Generate image.
    dlatents1 = Gs.components.mapping.run(latents1, None) # [seed, layer, component]
    images = Gs.components.synthesis.run(dlatents1, randomize_noise=False, **synthesis_kwargs)

    src_dlatents = np.load('./latent/mayuyu250px_01.npy')
    src_dlatents = src_dlatents.reshape(1,18,512)
    src_images = Gs.components.synthesis.run(src_dlatents, randomize_noise=False, **synthesis_kwargs)
    
    for i in range(1,101,4):
        dlatents = i/100*dlatents1+(1-i/100)*src_dlatents
        # Generate image.
        images = Gs.components.synthesis.run(dlatents, randomize_noise=False, **synthesis_kwargs)
        # Save image.
        os.makedirs(config.result_dir, exist_ok=True)
        png_filename = os.path.join(config.result_dir, 'example{}.png'.format(i))
        PIL.Image.fromarray(images[0], 'RGB').save(png_filename)

The only changes to this code since the last time are: The shape of the input data src_dlatents is matched.

    src_dlatents = np.load('./latent/mayuyu250px_01.npy')
    src_dlatents = src_dlatents.reshape(1,18,512)

In this way, the video posted above was obtained. For the time being, I will repost it below. This video is also reversed, so the video will be continuous and you can watch Mayuyu a little slowly.

1

By the way, the enlarged image that appears in this video is attached below, but do you know which one is the real one?

1	2	3

Summary

・ I learned the Style vector of the mapping latent space of my own image "Mayuyu". ・ I tried mixing using the obtained npy. ・ I tried "a woman transforms into Mayuyu"

・ Let's finally learn anime characters ・ Let's make a moving video