[PYTHON] I made a web application that converts photos into Van Gogh's style

This article is Request! Tips for developing on Azure using Python![PR] Microsoft Japan Advent Calendar 2020 This is the article on the 24th day.

Introduction

Using Flask, Pytorch and Azure, I created a web app ** that transforms photos into Van Gogh's style. I will introduce the procedure from environment construction to publication so that it will be an article that can be referred to when creating a web application using deep learning.

Web application finished product

** 1. When you open the website and send the image you want to process, **

↓ Selected image

** 2. An image converted to Van Gogh style will be returned. ** **

** You can access the site you actually created from the link below. Please play with it. ** ** Web application that converts images to Van Gogh style

• Since the image is resized to 256 x 256 and processed, the smaller image will look more like that.

By the way, here is a portrait of Van Gogh. Somehow the style of painting is similar ...

Overview of the created web application

I tried to summarize the outline of the created web application in the figure. ** This time, I used Azure, a Microsoft cloud service, as a web server within the scope of the free service (¥ 22,500 credit that can be used for 30 days after registration). ** The actual image processing part used CycleGAN implemented in Pytorch, and the appearance of the website was easily created only with HTML. Then, using Flask, which is a Python API, I created a link between Python and HTML.

What i did

The actual work and time required to create and publish a web application that implements the CycleGAN model are as follows.

1. WSL environment construction: 30 minutes
2. Build Python execution environment: 30 minutes
3. Selection of deep learning model: 2.5 hours
4. Azure Flask Tutorial: 30 minutes
5. Fixed CycleGAN code for web app: 2 hours
6. Web app deployment: 1 hour

Total: 7 hours

The hardest part was choosing a deep learning model and modifying the code for the web app. Last time had a hard time handling images in the web application, but this time there was no trouble around that, and the deployment work itself was completed in about an hour.

1. WSL environment construction (only for Windows)

Most machine learning libraries and deep learning models published on Github are developed on Ubuntu First on Linux. Therefore, in order to eliminate the extra work caused by the OS being Windows, we will build a WSL environment.

• Recently, WSL also supports NVIDIA GPU calculation, which enables high-speed deep learning model learning using WSL, making it extremely easy to use.

I think the Microsoft Official WSL Installation Guide ​​is the easiest way to install WSL.

I have selected the following Linux distributions:

When the installation is complete, you can start it by typing "wsl" at the command prompt or Windows Terminal.

• Although it is a small story, if you enter "wsl" in the address part of any folder, WSL will start with that as the current folder.

Check if the target Ubuntu has been installed with the following command.

ubuntu version check

> lsb_release -a

It seems that it was installed normally.

2. Build Python execution environment

We will install Anaconda to prepare the Python execution environment on WSL. (For Linux and Mac, just run the regular Anaconda installer) Regarding the installation of Anaconda on WSL, I think this article is simple and easy to understand.

By the way, please note that recently, Large-scale commercial use of Anaconda is charged.

• Reference: Anaconda package repository was charged for "large-scale" commercial use

After starting WSL, you can check the Python version by entering the following command.

python version check

$ python --version

It was installed successfully.

3. Selection of deep learning model

Choose a machine learning model to implement in your web app. When I was looking for a good GAN model on Github, I found CycleGAN. For those who are interested in theory, the treatise is here. If you use Github code, be careful about the license. The above code was a BSD License, so basically commercial use is OK.

• This is my personal opinion, but the readability of the implementation code is Pytorch> TensorFlow. We also recommend code that has many stars (300-) and few bugs.

By the way, the repository used this time had the following trained models. ・ Color monochrome images ・ Convert horses to zebras (I often see them) ・ Convert summer landscape to winter landscape

There is also a CycleGAN Demo Site (https://affinelayer.com/pixsrv/). This site will generate an original cat from the outline you drew.

4. Azure Flask tutorial

To deploy the selected machine learning model, use Azure App Service on Linux and Flask, the API of Python. Azure App Service on Linux is a service that allows you to deploy web apps on Linux very easily without having to build a troublesome environment.

After registering an Azure account, we will do the Official Tutorial to learn the flow of deployment.

** Hello, Azure! If you can create a site, it's OK for the time being. ** **

5. Fixed CycleGAN code for web apps

Next, modify the CycleGAN code published on Github to the Web application specifications. I think most deep learning model repositories have scripts for making inferences from input data using trained models. (Test.py file in the above repository) By modifying this, it will be used as an image processing function called from the main code of the web application.

The following three points were mainly corrected.

** How to write relative import ** I changed the folder structure and the name scope changed accordingly, so I fixed it. (Modify the module called by test.py)

** Remove and function unnecessary parts of inference code (test.py) ** Removed unnecessary parts such as model visualization code. I made it in the form of a function called from main.py. Along with that, the processing of pyorch Tensor type ⇒ numpy conversion of the data output by inference was added.

** Fix argparse ** argparse is a convenient library that allows you to easily implement the process of receiving arguments from the command line. It seems that it is mostly used in recent deep learning model implementations, but it is unnecessary when using it as a function of a web application, so I fixed it. (Delete unnecessary ones, specify all the rest with Default) You can also specify the CPU here, the model used, and the trained parameters.

I think that other deep learning models can be used if they are written in Pytorch or TensorFlow by modifying similar points.

6. Deploy web app

We will deploy the web application on Azure App Service on Linux. The folder structure is as follows.

My folder (any name is OK) 　├ static/ 　│　 └ CycleGAN model 　├ templates/ 　│　 └ index.html 　├ app.py 　└ requirements.txt

I will briefly describe each role. -** static : Enter the static data (CycleGAN model this time) that you want to access while the application is running. - templates : Save the html file. - app.py : A python file that describes data processing and cooperation with html. This is the first script to be executed in the app. - requirements.txt **: A file containing the python library and version required to run app.py. The environment is automatically built according to this at the time of deployment.

Now that you've done the tutorial and are ready to deploy your web app, just follow the one line below.

python version check

$ az webapp up --sku F1 --name <app-name>

By the way, you can select the machine level with the --sku argument. I can use F1 for free, but I chose S2 with 3.50 GB of RAM because I have free credit for ¥ 22,500. You can check the selectable options at here.

Source code

The deployed code is published in This Github repository. The trained weights could not be uploaded due to file size restrictions. If you want to actually run it, create cyclegan2gogh/static/checkpoints in the above folder and save the learned weights downloaded by here and it will work. Alternatively, you can prepare the training data and deploy the model you trained yourself.

reference

Building a GPU Deep Learning Environment on Ubuntu CUDA on WSL User Guide pytorch-CycleGAN-and-pix2pix Anaconda Terms of Service Quick Start: Create a Python App in Azure App Service on Linux (https://docs.microsoft.com/ja-jp/azure/app-service/quickstart-python?tabs=bash&pivots=python-framework-flask)

At the end

Thank you for reading to the end. It was quite a rush, but I managed to complete the process from environment construction to release in one day. If you don't mind, thank you for LGTM!