[PYTHON] I made a web application that converts photos into Van Gogh's style

This article is Request! Tips for developing on Azure using Python![PR] Microsoft Japan Advent Calendar 2020 This is the article on the 24th day.


Using Flask, Pytorch and Azure, I created a web app ** that transforms photos into Van Gogh's style. I will introduce the procedure from environment construction to publication so that it will be an article that can be referred to when creating a web application using deep learning.

Web application finished product

** 1. When you open the website and send the image you want to process, ** image.png

↓ Selected image images.jpg

** 2. An image converted to Van Gogh style will be returned. ** **


** You can access the site you actually created from the link below. Please play with it. ** ** Web application that converts images to Van Gogh style

By the way, here is a portrait of Van Gogh. Somehow the style of painting is similar ... image.png

Overview of the created web application

I tried to summarize the outline of the created web application in the figure. ** This time, I used Azure, a Microsoft cloud service, as a web server within the scope of the free service (¥ 22,500 credit that can be used for 30 days after registration). ** The actual image processing part used CycleGAN implemented in Pytorch, and the appearance of the website was easily created only with HTML. Then, using Flask, which is a Python API, I created a link between Python and HTML. image.png

What i did

The actual work and time required to create and publish a web application that implements the CycleGAN model are as follows.

  1. WSL environment construction: 30 minutes
  2. Build Python execution environment: 30 minutes
  3. Selection of deep learning model: 2.5 hours
  4. Azure Flask Tutorial: 30 minutes
  5. Fixed CycleGAN code for web app: 2 hours
  6. Web app deployment: 1 hour

Total: 7 hours

The hardest part was choosing a deep learning model and modifying the code for the web app. Last time had a hard time handling images in the web application, but this time there was no trouble around that, and the deployment work itself was completed in about an hour.

1. WSL environment construction (only for Windows)

Most machine learning libraries and deep learning models published on Github are developed on Ubuntu First on Linux. Therefore, in order to eliminate the extra work caused by the OS being Windows, we will build a WSL environment.

I think the Microsoft Official WSL Installation Guide ​​is the easiest way to install WSL.

I have selected the following Linux distributions:

OS environment Linux distribution
Windows10(64-bit) ver. 20H2(OS build 19042.685) Ubuntu 20.04 LTS

When the installation is complete, you can start it by typing "wsl" at the command prompt or Windows Terminal.

Check if the target Ubuntu has been installed with the following command.

ubuntu version check

> lsb_release -a

It seems that it was installed normally. image.png

2. Build Python execution environment

We will install Anaconda to prepare the Python execution environment on WSL. (For Linux and Mac, just run the regular Anaconda installer) Regarding the installation of Anaconda on WSL, I think this article is simple and easy to understand.

By the way, please note that recently, Large-scale commercial use of Anaconda is charged.

After starting WSL, you can check the Python version by entering the following command.

python version check

$ python --version

It was installed successfully. image.png

3. Selection of deep learning model

Choose a machine learning model to implement in your web app. When I was looking for a good GAN model on Github, I found CycleGAN. For those who are interested in theory, the treatise is here. If you use Github code, be careful about the license. The above code was a BSD License, so basically commercial use is OK.

By the way, the repository used this time had the following trained models. ・ Color monochrome images ・ Convert horses to zebras (I often see them) ・ Convert summer landscape to winter landscape

There is also a CycleGAN Demo Site (https://affinelayer.com/pixsrv/). This site will generate an original cat from the outline you drew. image.png image.png

4. Azure Flask tutorial

To deploy the selected machine learning model, use Azure App Service on Linux and Flask, the API of Python. Azure App Service on Linux is a service that allows you to deploy web apps on Linux very easily without having to build a troublesome environment.

After registering an Azure account, we will do the Official Tutorial to learn the flow of deployment.

** Hello, Azure! If you can create a site, it's OK for the time being. ** **

5. Fixed CycleGAN code for web apps

Next, modify the CycleGAN code published on Github to the Web application specifications. I think most deep learning model repositories have scripts for making inferences from input data using trained models. (Test.py file in the above repository) By modifying this, it will be used as an image processing function called from the main code of the web application.

The following three points were mainly corrected.

** How to write relative import ** I changed the folder structure and the name scope changed accordingly, so I fixed it. (Modify the module called by test.py)

** Remove and function unnecessary parts of inference code (test.py) ** Removed unnecessary parts such as model visualization code. I made it in the form of a function called from main.py. Along with that, the processing of pyorch Tensor type ⇒ numpy conversion of the data output by inference was added.

** Fix argparse ** argparse is a convenient library that allows you to easily implement the process of receiving arguments from the command line. It seems that it is mostly used in recent deep learning model implementations, but it is unnecessary when using it as a function of a web application, so I fixed it. (Delete unnecessary ones, specify all the rest with Default) You can also specify the CPU here, the model used, and the trained parameters.

I think that other deep learning models can be used if they are written in Pytorch or TensorFlow by modifying similar points.

6. Deploy web app

We will deploy the web application on Azure App Service on Linux. The folder structure is as follows.

My folder (any name is OK)  ├ static/  │  └ CycleGAN model  ├ templates/  │  └ index.html  ├ app.py  └ requirements.txt

I will briefly describe each role. -** static : Enter the static data (CycleGAN model this time) that you want to access while the application is running. - templates : Save the html file. - app.py : A python file that describes data processing and cooperation with html. This is the first script to be executed in the app. - requirements.txt **: A file containing the python library and version required to run app.py. The environment is automatically built according to this at the time of deployment.

Now that you've done the tutorial and are ready to deploy your web app, just follow the one line below.

python version check

$ az webapp up --sku F1 --name <app-name>

By the way, you can select the machine level with the --sku argument. I can use F1 for free, but I chose S2 with 3.50 GB of RAM because I have free credit for ¥ 22,500. You can check the selectable options at here.

Source code

The deployed code is published in This Github repository. The trained weights could not be uploaded due to file size restrictions. If you want to actually run it, create cyclegan2gogh/static/checkpoints in the above folder and save the learned weights downloaded by here and it will work. Alternatively, you can prepare the training data and deploy the model you trained yourself.


Building a GPU Deep Learning Environment on Ubuntu CUDA on WSL User Guide pytorch-CycleGAN-and-pix2pix Anaconda Terms of Service Quick Start: Create a Python App in Azure App Service on Linux (https://docs.microsoft.com/ja-jp/azure/app-service/quickstart-python?tabs=bash&pivots=python-framework-flask)

At the end

Thank you for reading to the end. It was quite a rush, but I managed to complete the process from environment construction to release in one day. If you don't mind, thank you for LGTM!

Recommended Posts

I made a web application that converts photos into Van Gogh's style
I made a web application in Python that converts Markdown to HTML
I made a WEB application with Django
I made a web application that maps IT event information with Vue and Flask
I tried a tool that imitates Van Gogh's pattern with AI
I tried benchmarking a web application framework
I made a web application that graphs the life log recorded on Google Home like a Gantt chart.
I made a GUI application with Python + PyQt5
I want to create a web application that uses League of Legends data ①
A site that converts curl commands into requests
I made a VM that runs OpenCV for Python
I made an Android application that displays Google Map
A Python program that converts ical data into text
〇✕ I made a game
I made a tool to automatically generate a state transition diagram that can be used for both web development and application development
I tried "Streamlit" which turns the Python code into a web application as it is
A story that stumbled when I made a chatbot with Transformer
A story that I had a hard time trying to create an "app that converts images like paintings" with the first web application
I made a LINE BOT that returns parrots with Go
AnimeGAN Yaru (Windows10, Python3.6) that converts live-action into anime style
White-box Cartoonization that converts live-action into anime style (Windows10, Python3.6)
I made a simple book application with python + Flask ~ Introduction ~
I made a class that easily performs unixtime ← → datetime conversion
I made a fucking app that won't let you skip
I made a rigid Pomodoro timer that works with CUI
I made a plug-in that can "Daruma-san fell" with Minecraft
I made you to execute a command from a web browser
I made a neural network generator that runs on FPGA
Note that I was addicted to accessing the DB with Python's mysql.connector using a web application.
I made a python text
I made a discord bot
Let's turn PES analysis software into a WEB application! First step!
[Python3] I made a decorator that declares undefined functions and methods.
I made a package that can compare morphological analyzers with Python
I made a program that solves the spot the difference in seconds
I made a web server with Raspberry Pi to watch anime
I wanted to convert my face photo into a Yuyushiki style.
I made a Twitter bot that mutters Pokemon caught by #PokemonGO
I made a shuffle that can be reset (reverted) with Python
Map Creator, a service that converts latitude and history into URLs
I made a lo command that is more useful than ls
I made a slack bot that notifies me of the temperature
I made a scaffolding tool for the Python web framework Bottle
I made a library that adds docstring to a Python stub file.
I made a program that automatically calculates the zodiac with tkinter
[python] I made a class that can write a file tree quickly