[PYTHON] I tried to make a motion detection surveillance camera with OpenCV using a WEB camera with Raspberry Pi

Thing you want to do

It seems that suspicious people are appearing at the entrance of the apartment where I live, so I attached a WEB camera to the entrance door and made a system to analyze and record with Raspberry Pi. The specifications are as follows.

--Display the captured video and time on the screen --Record a still image in jpeg format when someone comes in front of the entrance --Embed the shooting time in the recorded still image --False detection is tolerated to some extent

• By the way, permission has been obtained for the installation of the camera.

Referenced articles

At first, I decided to do face detection by referring to the article of PC Studio, but there are many false positives and it is not suitable for this application. It was.

https://www.pc-koubou.jp/magazine/19205

As a result of various thoughts, I thought that it could be realized by motion detection (detection that there was a change in the image being shot), and as a result of trial and error, it went well, so I will introduce the method.

In addition, I referred to this article for how to detect motion.

https://qiita.com/K_M95/items/4eed79a7da6b3dafa96d

Things necessary

--Raspberry Pi 3 Model B Plus Rev 1.3 (Any Raspberry Pi with USB stick?) --USB connected webcam (maybe anything, use a fairly old one) --Python (this time using v2.7) --OpenCV (v3.2.0 is used this time)

For the installation procedure of OpenCV, refer to the above article PC Studio. Python uses the original one.

Construction procedure

1. Raspberry Pi setup
2. Set up VNC environment as needed
3. Install OpenCV
4. Connect the WEB camera
5. Check if the WEB camera is recognized
6. Write and run the program

WEB camera recognition confirmation

It seems that the driver is included in the USB-connected WEB camera from the beginning, so if you connect it, it will be recognized.

$ lsusb
Bus 001 Device 006: ID 0c45:62e0 Microdia MSI Starcam Racer
Bus 001 Device 005: ID 0424:7800 Standard Microsystems Corp. 
Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

It is OK if the name like a USB camera is displayed in the result of the lsusb command. If you don't know, check if the number of items increases when you unplug and connect.

Source code

The code I wrote is as follows.

security_cam_motion.py

 # -*- coding: utf-8 -*-
import time
import datetime
import cv2 as cv

#A program that realizes a surveillance camera using a WEB camera
#Motion detection, save the jpg file with the date and time embedded at that time


#Directory to save images
save_dir  = './image/'

#The file name should be a character string including the date and time
#Specify the file name to be added after the date and time
fn_suffix = 'motion_detect.jpg'

#Create an instance of VideoCapture.
cap = cv.VideoCapture(0) 

#Specify vertical and horizontal resolution
cap.set(cv.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv.CAP_PROP_FRAME_HEIGHT, 480)

#Pixel value when binarized
DELTA_MAX = 255

#Threshold for detecting changes in each dot
DOT_TH = 20

#Motion factor(How many points have changed)But
#How much or more should you record?
MOTHON_FACTOR_TH = 0.20

#Store data for comparison
avg = None

while True:
    
    ret, frame = cap.read()     #Read 1 frame
    motion_detected = False     #Flag indicating whether movement was detected

    dt_now = datetime.datetime.now() #Time when the data was acquired

    #File name and date and time to embed in the image
    dt_format_string = dt_now.strftime('%Y-%m-%d %H:%M:%S') 
    f_name = dt_now.strftime('%Y%m%d%H%M%S%f') + "_" + fn_suffix


    #Make it monochrome
    gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
    
    #Get a frame for comparison
    if avg is None:
        avg = gray.copy().astype("float")
        continue
        
    
    #Calculate the difference between the current frame and the moving average
    cv.accumulateWeighted(gray, avg, 0.6)
    frameDelta = cv.absdiff(gray, cv.convertScaleAbs(avg))

    #Threshold processing the delta image
    thresh = cv.threshold(frameDelta, DOT_TH, DELTA_MAX, cv.THRESH_BINARY)[1]

    #Calculate the motion factor. How much has changed as a whole?
    motion_factor = thresh.sum() * 1.0 / thresh.size / DELTA_MAX 
    motion_factor_str = '{:.08f}'.format(motion_factor)
    
    #Write date and time on the image
    cv.putText(frame,dt_format_string,(25,50),cv.FONT_HERSHEY_SIMPLEX, 1.5,(0,0,255), 2)
   #Motion to image_Write factor value
    cv.putText(frame,motion_factor_str,(25,470),cv.FONT_HERSHEY_SIMPLEX, 1.5,(0,0,255), 2)
    
    #If the motion factor exceeds the threshold value, motion is detected.
    if motion_factor > MOTHON_FACTOR_TH:
        motion_detected = True
    
    #Save the image if motion is detected
    if motion_detected  == True:
        #save
        cv.imwrite(save_dir + f_name, frame)
        print("DETECTED:" + f_name)
    
    
    #From here, processing the image to be displayed on the screen
    #Add a contour to the threshold of the image
    image, contours, hierarchy = cv.findContours(thresh.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
    frame = cv.drawContours(frame, contours, -1, (0, 255, 0), 3)
    
    
    #Display the resulting image
    cv.imshow('camera', frame)


    #Wait until any key is pressed
    k = cv.waitKey(1000)  #The argument is the wait time(ms)
    if k == 27: #Ends when Esc is input
        break
    

print("Bye!\n")
#Close the displayed window
cap.release()
cv.destroyAllWindows()

Start command

$ sudo python security_cam_motion.py

Execution result

As a result of executing it, it looks like this.

If there is no change, nothing happens. The number at the bottom is the motion factor. Since there is no change, it is 0.0.

When a person appears in front of the entrance, the change is recognized and the jpeg file is saved. Since the motion factor is above a certain level, you can see that the movement has been detected.

The border is not displayed in the saved jpeg file.

About motion factor

Motion factor is a coined word that I coined by myself. A factor that indicates how much of the pixels on the entire screen have changed.

It is called motion detection, but the principle is that we are looking for pixels that are different from the previous frame. A slight change in the environment may detect a slight difference from the previous frame. If it is detected each time, the surveillance camera will not be useful. On the other hand, if a person stands in front of the camera, a significant percentage of the pixels should change. If you decide to record when a certain percentage changes, it seems that you can achieve the purpose of "recording when a person comes in front of the entrance".

About false positives

Think about false positive false positives. In other words, how to deal with "detecting when no one is coming". People rarely come because the place where the surveillance camera is installed is just a room in the condominium. So it's a problem if the data is filled with false positives. Think about ways to prevent false positives. It seems that how much change should be ... can be adjusted by the motion factor and the second argument of the cv.threshold () function (threshold value of change from the previous image of each dot). Since the DOT_TH variable is used as the second argument, adjust with this variable. If the value is too small, it will be falsely detected due to changes in the environment or pixel fluctuations. Even so, some pixels occasionally falsely detect changes even without actual movement, so the motion factor prevents false positives.

   : 
#Threshold for detecting changes in each dot
DOT_TH = 20
#Motion factor(How many points have changed)But
#How much or more should you record?
MOTHON_FACTOR_TH = 0.20
   :
#Threshold processing the delta image
thresh = cv.threshold(frameDelta, DOT_TH, DELTA_MAX, cv.THRESH_BINARY)[1]
#Calculate the motion factor. How much has changed as a whole?
motion_factor = thresh.sum() * 1.0 / thresh.size / DELTA_MAX 
motion_factor_str = '{:.08f}'.format(motion_factor)
   :

About false positives (false negatives)

On the other hand, "there is a person but it is not recorded" is not useful as a surveillance camera. Probably, in this system, there are people, but the cases that are not recorded are "wearing clothes with the same color scheme as the wall", "cut off at the edge of the screen", "wearing optical camouflage". I will. In this case, I decided to tolerate it because I don't know what is in the picture and it may not be useful. There is no point in seeking perfection. Balance is important.

that's all