[PYTHON] I tried to easily create a high-precision 3D image with one photo [1]. (Depth can now be edited in PNG.)
Overview
** One photo ** ** High-precision 3D image ↓ Easy creation **.
(↓ It doesn't make any sense, but this pizza sharpens the PNG of depth. There is a part where the image is dragged, but I wonder if it is a pizza ...)
Below are the papers and github.
https://arxiv.org/pdf/2004.04727.pdf Paper "3D Photography using Context-aware Layered Depth Inpainting" Meng-Li Shih1 and others
github is below. https://github.com/vt-vl-lab/3d-photo-inpainting
Paper abstract (Google Translate mom)
738/5000 We suggest how to convert a single RGB-D input image to a 3D photo. This is a new view composition multi-layer representation that includes hallucinatory color and depth structures in areas blocked by the original view. Using a layered depth image with explicit pixel connections as a basic representation, we present a learning-based repair model that synthesizes new local color and depth content into a closed area in a spatial context-aware way. .. The resulting 3D photo can be efficiently rendered with motion parallax using a standard graphics engine. Verify the effectiveness of the method in a variety of challenging everyday situations and reduce artifacts compared to state-of-the-art technology.
It will be an additional note, 2020/05/23 ** About 3 days ago, depth could be edited in PNG. ** ** I tried it.
I tried it (work)
(The code on github worked easily in my windows environment. I may need a GPU. I don't need a good GPU. The processing time was a minute or two.)
As a creation procedure, Put the image you want to process in the image folder Just type the following command.
python main.py --config argument.yml
For publication here, the original photo is "High Quality Free Image Material" https://pixabay.com/ja/ Obtained from. (Similar to King Kong in the overview)
Move some objects from one photo? Since it is (shaking), of course, there is a part where there is no information **, but it seems that it is forcibly generated. great.
** I'm sorry, it may take about 30 seconds to display. .. .. ** **
** All inputs are one still image ** 1 sheet!
I tried it (additional work)
** Metropolitan Museum of Art ** Use the paintings in the collection.
I tried it (failure, CG work, etc.)
Again, the original photo is "high quality free image material" https://pixabay.com/ja/ Obtained from.
↓ ** Failure? **. (Rather, even when computers are incompatible ...)
↓ ** Failure **. ** Escher system . Also, balloons are difficult, aren't they? You may name it " Balloon Problem **". However, the picture is beautiful. .. ..
↓ Somehow the texture. .. .. Failure.
↓ Normal failure.
(2020/05/23) About 3 days ago, depth could be edited with PNG.
I edited the depth.
Step 1: In argument.yml Change depth_format to png and execute.
・ ・ ・ depth_format: '.png' ・ ・ ・
Then the depth will be saved as PNG. (I don't want to display this png ↓ in a large size at all, but ... It is probably displayed in a large size along the road ...)
Step 2:
Edit PNG.
(At the bottom of the car, make the part where the depth is shallow deeper (= black).)
See the results shown below to see if such a rough edit is okay.
Step 3:
require_Set midas to False so that the edited PNG is used.
require_midas: False
Then re-execute the conversion.
As intended, the depth at the bottom of the car is deeper. However, it looks like PNG,
Rough PNG editing didn't clean it up.
If you can do paint processing normally, I think it's okay. I can't. With a brush
I painted it properly. .. ..
You may not be able to tell what you have done because the work is complicated, but when you are dissatisfied with the automatically detected depth, you can edit the depth expressed as PNG as an image.
#Information that may be useful
・ GFORCE GTX1050Ti (Dedicated GPU memory 4).0GB)Then
**It may not work due to insufficient memory of GPU**。
this is,**Rather than relying on image size**、
Perhaps it's related to the amount of memory required to represent the 3D in the image content or something.
I haven't found a solution. It may be possible to manage by changing parameters.
Red car, so dedicated GPU memory 3.About 4GB, white one is 2.About 7GB was used.
For this memory problem, why not use Google Colab for the time being?
-I showed editing depth information with PNG, but surely the default is numpy?。
That data(numpy)If you can handle the rattling, you may edit it.
#Summary
I don't understand how great it is technically.
However, I wonder if it feels like a force.
(In the first place, people's perception when looking at a photo is also forcible, so ...)
I will study technology.
(For the time being, I understand the habit of processing, so I think I can make a little more fun work.)
At this stage, I have no idea what it can be used for.
**Please try it yourself. "Indoor selfie" is recommended**
#Summary 2
Described after "I tried it (additional work)".
Unfortunately,**I got tired of seeing it in no time. .. .. ..**
It's tough if you don't have a purpose and output for it. .. .. ..
mere**effect**I mean. .. .. Then you get tired of it. Furthermore, we expect the next level of technology.
I'm tired of it, but I'm interested in technology, though I don't understand it yet. .. ..
It was easy to edit the depth.
#from now on
I think it's better to display the name of the author of pixabay, separately.
Is there a different way to show the processed image? ?? ?? I think it's simple.
If you have any comments, please.
reference.
[I tried to easily create a high-precision 3D image with one photo [2].(Try processing depth with numpy)](https://qiita.com/torinokaijyu/items/e761c00c87d6a00b8c30)
[I tried to easily create a high-precision 3D image with one photo [3]. Only depth without permission feat. intel-of isl.](https://qiita.com/torinokaijyu/items/6889598b732851c2e8fd)
Recommended Posts