[PYTHON] It is difficult to install a green screen, so I cut out only the face and superimposed it on the background image

I detected my face from the webcam and tried to overlay only the face on the background image. If you pass the video as a webcam to the web conference application, you can easily appear in the web conference. You can also be a tree sage who wanted to be (?) When you were little.

Note) There is no additional function for blindfolding. The blindfold was added to the screenshot later by hand for this article.

Task

Since the beginning of March, I have been at home for web conferencing. The room was so dirty that I managed to do it with no video, just audio and screen sharing. However, I feel that the web conferencing alone is breaking down the trust savings that I have had so far. I'm very happy to see someone who sometimes shows my face. I also want to show up. But the room is too small and dirty!

Usually you'll give up and clean up the room, or set up a green screen and make a video with chroma key composition. But I can't get rid of it. .. ..

The web conferencing system widely used in the company is WebEx. There is no background conversion function like Zoom, and I don't feel like developing it immediately like Teams. .. .. I use Zoom occasionally, but the machine power is not enough and a green screen is essential. But there is no place to install a green screen. .. ..

Delusion

By the way, Zoom cuts out not only the face but also the hands and upper body from the background. So is heavy processing necessary? ?? If you divide it only by face detection, you should be able to do it without so much load even on a weak PC! ??

Concept

Like this.

Webcam-> Face area detection with OpenCV-> Extract only face area and overlay it on background image-> Output as webcam image-> Used with WebEx and Zoom

Looking at this blog post, it seems that Face detection can be done immediately with OpenCV. .. I've done cutting and overlaying at the Python Imaging Library (Pillow). If it can be processed about 5 times per second, it should be enough video to be used in a web conference. Now, what should I do to output the created image as a webcam image? OpenCV does not seem to have such a function. I couldn't find a library that can be used on Mac or Windows from Python.

It's not very smart, but on MacOS, it seems that CamTwist can capture a part of the screen and let other software recognize it as a webcam and pass it on.

I tried it

I have uploaded the code to GitHub. The part I wrote is Python, but I use OpenCV and Qt. Since it is in GW, it has not been put into actual battle yet. I wonder if it will be the next web conference soon.