Hello
I recently started studying machine learning, so I'll write it for myself. Since this is my first time writing, I would be grateful if you could point out any mistakes in the writing style.
This time, I tried to move TecoGAN, which can perform super-resolution of videos. Source: https://github.com/thunil/TecoGAN Paper: https://arxiv.org/pdf/1811.09393.pdf
I don't understand it well, so if you have any questions, please let me know ... Also, it may be fixed in the future. Also, the OS uses ubuntu. TecoGAN uses wget for something, so it's annoying for windows.
OS : Ubuntu 18.04 CPU: i5 4670k GPU: Geforce 2060SP Memory: 16GB Language: python 3.6
Basically follow the git procedure. First, get the source code from git.
cmd
git clone https://github.com/thunil/TecoGAN.git
Then install the required packages.
cmd
pip3 install tensorflow-gpu==1.14.0
# tensorflow-Install gpu. If you don't use gpu, only tensorflow is here.
pip3 install -r requirements.txt
#Installation of other required packages
It differs from git in that it installs tensorflow-gpu, but this is because the version of tensorflow-gpu will be 2.x if you follow the git procedure, and the program will malfunction due to the version difference.
You may also be relieved to build a virtual environment and install packages there. I didn't build it because I started TecoGAN first.
I used GPU to calculate TecoGAN, so I put in CUDA and cudnn. [CUDA cudnn compatible version quick reference table] https://www.tensorflow.org/install/source?hl=ja#gpu_support_2 The above site is a quick reference of the versions of tensorflow, CUDA and cudnn that work properly. If you do not observe this, the GPU may not be recognized.
When I check tensorflow-gpu == 1.14.0, CUDA10.0 and cudnn7.4 seem to work normally.
cmd
$ wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64
$ pkg -i cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64
$ apt-key add /var/cuda-repo-10-0-local-10.0.130-410.48/7fa2af80.pub
$ apt-get install cuda-toolkit-10-0
That's it.
cmd
nvcc -V
If you type the above command, the CUDA version will be displayed.
It's a little annoying. Because you have to register as a member. https://developer.nvidia.com/cudnn Select [Download cuDNN] from the above link, and then register with [Join now].
After that, after logging in, check [I Agree ~~] and select the version of cudnn you want. In this case, it is cudnn v7.4.2.
Select cudnn Library for Linux and wait for the download to finish. After that, go under the downloaded folder, unzip it, and install it in the environment.
cmd
$ tar -xvzf cudnn-10.0-linux-x64-v7.4.2.24.tgz
$ cp -a cuda/lib64/* /usr/local/cuda/lib64/
$ cp -a cuda/include/* /usr/local/cuda/include/
$ cp -a cuda/include/* /usr/include/
That's it. For cudnn, using wget causes an error. Maybe because I don't log in.
As a memo about the driver
cmd
ubuntu-drivers devices
Multiple recommended drivers appear Install the driver labeled [recommended]
cmd
sudo apt install nvidia-driver-xxx
After rebooting, the nvidia-smi command will be available. You can see the usage rate of gpu and the device of gpu.
You can check it with the following two lines of code.
python
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
The GPU recognizes if the following words are written at the bottom.
Created TensorFlow device (/device:GPU:0 with 6654 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2060 SUPER, pci bus id: 0000:01:00.0, compute capability: 7.5)
TecoGAN has a trained model, and the program is operated around it. So, install the trained model, low resolution video and high resolution video. It may have taken about 10 minutes.
cmd
python runGan.py 0
After the installation is complete, run it with the following code.
cmd
python runGan.py 1
A super-resolution calendar image is generated. The results are in the results folder under the TecoGAN folder.
After all it is output as an image. Other tools are needed to encode, and TecoGAN recommends ffmpeg.
sudo apt-get install ffmpeg
Please install it and compare it with a video. It seems to be pretty clean.
By the way
python runGan.py 2
You can also evaluate numerically with. The index is PSNR SSIM LRIPS tLP100 tOF That's right, but I still don't understand it properly. Especially the bottom three Please let me know.
It's boring to move the test image. First, make your favorite video low resolution.
ffmpeg -i video path.mp4 -vf scale=270:-1 Low resolution video output path.mp4
In scale, if you specify the number of pixels on one side, it will automatically adjust to the same aspect ratio as the original video. I have the original video 1920 * 1080, so the reduced resolution video is 480 * 270.
Because the magnification of TecoGAN is fixed at 4 times, this resolution is used to compare the super-resolution video with the original video.
Next, use ffmpeg etc. to make a low-resolution video image for each frame.
ffmpeg -i video path.mp4 -vcodec png Folder to output images/%04d.png
Now, if you make your favorite video into a frame group, it will be super-resolution.
Create a folder in TecoGAN / LR with a name of your choice.
This time, I chose hogehoge as appropriate.
Put your frame group in TecoGAN / LR / hogehoge.
Next, check the program of runGan.py.
runGan.py
elif( runcase == 1 ): # inference a trained model
dirstr = './results/' # the place to save the results
# testpre = ['calendar'] # the test cases
testpre = ['hogehoge'] # the test cases #add to
if (not os.path.exists(dirstr)): os.mkdir(dirstr)
Add a one-line program. Now you are ready to go.
python runGan.py 1
If you move it, a super-resolution frame group should be generated.
Check results / hogehoge.
Probably, I think that the result is not so good. You have to train the model. Next, I would like to make a note about learning the model in the future. I was tired of writing a lot for the first time. I would appreciate it if you could point out any points that are difficult to see.
Super-resolution to full HD is not possible if the GPU is sloppy.
You will run out of memory.
So Google colaboratory is very useful.
I've just started touching it recently, but apparently it's easier to work with google drive.
So I will touch it a little to make it work. It's just a memo, so I'm sorry if it's hard to see.
First of all, google colaboratory allows you to use high spec GPUs for free. Specifically, just select "Runtime"-> "Change Runtime Type"-> "None"-> "GPU". Very easy.
In google cola boratory, various packages necessary for machine learning are included from the beginning. Of course, the latest version of tensorflow has also been introduced.
But with TecoGAN, tensorflow doesn't work unless it's version 1.x. (It works, but it throws a lot of errors)
So, lower the version first.
!pip uninstall -y tensorflow
!pip install tensorflow-gpu==1.14.0
In google cola boratoty, commands basically add "!" At the beginning of the sentence. As I said earlier, I lowered the version of tensorflow, which causes problems with the versions of cuda and cudnn and makes the GPU unrecognizable.
So please reinstall CUDA and cudnn.
!apt-get --purge remove cuda*
!apt autoremove
The above command removes all cuda. Same as usual. It seems that cudnn does not have to be erased. However, after installing cudnn, you may want to run the following program.
import os
os.environ["CUDA_HOME"]="/usr/local/cuda"
os.environ["LIBRARY_PATH"]="/usr/local/cuda/lib64"
I'm sorry if there is a problem.
Just in case, restart the runtime and check the version of tensorflow.
!pip freeze | grep -i -e tensorflow
--
tensorflow-addons==0.8.3
tensorflow-datasets==2.1.0
tensorflow-estimator==1.14.0
tensorflow-gcs-config==2.3.0
tensorflow-gpu==1.14.0
tensorflow-hub==0.8.0
tensorflow-metadata==0.22.2
tensorflow-privacy==0.2.2
tensorflow-probability==0.11.0
Looks good. Check the GPU.
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
---
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 7140650767936614436, name: "/device:XLA_GPU:0"
device_type: "XLA_GPU"
memory_limit: 17179869184
locality {
}
incarnation: 10124152797065672754
physical_device_desc: "device: XLA_GPU device", name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 8094731668087896846
physical_device_desc: "device: XLA_CPU device", name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 15956161332
locality {
bus_id: 1
links {
}
}
incarnation: 845827148846290348
physical_device_desc: "device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0"]
It is recognized. After that, please play with TecoGAN in the same way.
However, I feel that I couldn't drop the video (vimeo) necessary for learning the model.
!python3 dataPrepare.py --start_id 2000 --duration 120 --REMOVE --disk_path TrainingDataPath #I feel like I couldn't do it ...?
It may be a little difficult because you will have to drop the video yourself and divide it into frames. Well, if you want to super-resolution your favorite video beautifully, it may be a road that you have to go through, so it may not be a problem.
Please point out things such as immobility and typographical errors.
Also, I'm currently struggling to double, triple, or 6x the super-resolution of TecoGAN, but it's not working. This is because our FRVSR is a 4x model in the first place. I would appreciate any advice.
Thank you very much.
Recommended Posts