[PYTHON] [Introduction to StyleGAN] I played with "A woman transforms into Mayuyu" ♬

You can now learn the Style vector of the mapping latent space of your own image. So, I played with "A woman transforms into Mayuyu". This time, it was necessary to install dlib for self-image learning, and I was able to install it safely by following the reference ①. As a result, the Style vector $ w $ of Mayuyu's latent space could be obtained, and the following transformation video was created. simple_method10_1_dr1024.gif

【reference】 ① When you get stuck with dlib installation on windows

What i did

・ Installation of dlib ・ Pre-processing with align_images.py ・ Mayuyu and mixing ・ Try "a woman transforms into Mayuyu"

・ Installation of dlib

StyleGAN requires image pre-processing before learning. The code in align_images.py has the following description.

Extracts and aligns all faces from images using DLib 
and a function from original FFHQ dataset preparation step
    python align_images.py /raw_images /aligned_images

This means that all face images need to be pre-processed in align_images.py. And it was a difficult place to install this Dlib. However, as shown in Reference ① above, it worked fine with the following procedure.

・ Pre-processing with align_images.py

All you have to do is put the raw data in / raw_imges and run the following command:

python align_images.py /raw_images /aligned_images

As a result, the following aligned_image is output.

raw_image aligned_image
size 250x235 1024x1024
image mayuyu250px.jpg mayuyu250px_01.png

At this stage, the face part of the image is cut out and enlarged to 1024x1024. Using this image, the Style vector mayuyu.npy of the mapping latent space is learned by the following command and stored in latent /.

python encode_images.py aligned_images/ generated_images/ latent/

・ Mayuyu and mixing

Using the Style vector of this latent space, you can mix with various images as you did last time. See below for the code. StyleGAN/mayuyu_mix.py However, the changes to $ w $ in StyleGAN are as follows.

style_ranges=[range(1,18)]+[range(2,18)]+[range(3,18)]+[range(4,18)]+[range(5,18)]+[range(6,18)]

The results are as follows. mayuyu250px-style-mixing_case5_6.jpg It is interesting that a pretty beautiful woman appears by arranging with a man. As a whole, men below range (4,18) are becoming feminine, and other people are beginning to have more Mayuyu-side characteristics than the original. In other words, range (0,3) takes over a considerable amount of global features in Mayuyu.

・ Try "a woman transforms into Mayuyu"

So, at the end, I will try mixing instead of exchanging. The main parts of the code are: The whole is placed below. StyleGAN/simple_dlatents_mixing2.py

simple_dlatents_mixing2.py


    # Pick latent vector.
    rnd = np.random.RandomState(6) #5
    latents1 = rnd.randn(1, Gs.input_shape[1])
    print(latents1.shape)
    # Generate image.
    dlatents1 = Gs.components.mapping.run(latents1, None) # [seed, layer, component]
    images = Gs.components.synthesis.run(dlatents1, randomize_noise=False, **synthesis_kwargs)

    src_dlatents = np.load('./latent/mayuyu250px_01.npy')
    src_dlatents = src_dlatents.reshape(1,18,512)
    src_images = Gs.components.synthesis.run(src_dlatents, randomize_noise=False, **synthesis_kwargs)
    
    for i in range(1,101,4):
        dlatents = i/100*dlatents1+(1-i/100)*src_dlatents
        # Generate image.
        images = Gs.components.synthesis.run(dlatents, randomize_noise=False, **synthesis_kwargs)
        # Save image.
        os.makedirs(config.result_dir, exist_ok=True)
        png_filename = os.path.join(config.result_dir, 'example{}.png'.format(i))
        PIL.Image.fromarray(images[0], 'RGB').save(png_filename)

The only changes to this code since the last time are: The shape of the input data src_dlatents is matched.

    src_dlatents = np.load('./latent/mayuyu250px_01.npy')
    src_dlatents = src_dlatents.reshape(1,18,512)

In this way, the video posted above was obtained. For the time being, I will repost it below. This video is also reversed, so the video will be continuous and you can watch Mayuyu a little slowly.

1 simple1_.png
simple3_.png simple_method10_1_dr.gif

By the way, the enlarged image that appears in this video is attached below, but do you know which one is the real one?

1 2 3
example5.png example1.png example9.png

Summary

・ I learned the Style vector of the mapping latent space of my own image "Mayuyu". ・ I tried mixing using the obtained npy. ・ I tried "a woman transforms into Mayuyu"

・ Let's finally learn anime characters ・ Let's make a moving video

Recommended Posts

[Introduction to StyleGAN] I played with "A woman transforms into Mayuyu" ♬
[Introduction to StyleGAN] I played with "The Life of a Man" ♬
[Introduction to StyleGAN] I played with style_mixing "Woman who takes off glasses" ♬
[Introduction to system trading] I drew a Stochastic Oscillator with python and played with it ♬
[Introduction to sinGAN-Tensorflow] I played with the super-resolution "Challenge Big Imayuyu" ♬
[Introduction to Matplotlib] Axes 3D animation: I played with 3D Lissajous figures ♬
[Introduction] I want to make a Mastodon Bot with Python! 【Beginners】
[Introduction to RasPi4] I played with "Hiroko / Hiromi's poisonous tongue conversation" ♪
[Introduction to AWS] I played with male and female voices with Polly and Transcribe ♪
I want to INSERT a DataFrame into MSSQL
[Introduction to StyleGAN2] Independent learning with 10 anime faces ♬
I want to make a game with Python
I played with Diamond, a metrics collection tool
I want to write to a file with Python
I tried to create a table only with Django
I want to transition with a button in flask
I want to climb a mountain with reinforcement learning
I made a program to convert images into ASCII art with Python and OpenCV
I played with wordcloud!
I tried to draw a route map with Python
I want to split a character string with hiragana
[Introduction to Pytorch] I tried categorizing Cifar10 with VGG16 ♬
I want to manually create a legend with matplotlib
I want to run a quantum computer with Python
[Introduction to AWS] I tried playing with voice-text conversion ♪
I want to bind a local variable with lambda
I tried to implement a volume moving average with Quantx
I tried to automatically create a report with Markov chain
I made a package to filter time series with python
I want to start a jupyter environment with one command
I made a simple book application with python + Flask ~ Introduction ~
I want to make a click macro with pyautogui (desire)
[Markov chain] I tried to read a quote into Python.
I tried to solve a combination optimization problem with Qiskit
I want to make a click macro with pyautogui (outlook)
I want to use a virtual environment with jupyter notebook!
I want to install a package from requirements.txt with poetry
I tried to sort a random FizzBuzz column with bubble sort.
[Visualization] I want to draw a beautiful graph with Plotly
I tried to divide the file into folders with Python
I tried to translate English subtitles into Japanese with Udemy
I tried to divide with a deep learning language model
A quick introduction to pytest-mock
Introduction to Nonlinear Optimization (I)
I made a library to easily read config files with Python
[Introduction to Python] How to split a character string with the split function
A memorandum when I tried to get it automatically with selenium
I wanted to create a smart presentation with Jupyter Notebook + nbpresent
I wanted to convert my face photo into a Yuyushiki style.
[3rd] I tried to make a certain authenticator-like tool with python
[Python] A memo that I tried to get started with asyncio
I wrote a script to get you started with AtCoder fast!
I want to do a full text search with elasticsearch + python
I tried to make a periodical process with Selenium and Python
I tried to make a 2channel post notification application with Python
I read "Reinforcement Learning with Python: From Introduction to Practice" Chapter 1
I wanted to solve the ABC164 A ~ D problem with Python
Let's feel like a material researcher with python [Introduction to pymatgen]
I tried to create Bulls and Cows with a shell program
[Introduction to StyleGAN] Unique learning of anime with your own machine ♬
[4th] I tried to make a certain authenticator-like tool with python