[PYTHON] Accelerate read by specifying OpenCV frame

1.First of all

When using mp4 with OpenCV, there was a phenomenon that it was slow to specify the frame position with set and read. Since it took too long to acquire the video at regular intervals (every 0.5 seconds), the countermeasures are described below. (If you have any other better means, please let me know in the comments.)

2. For readers

・ Those who use video data at intervals larger than one frame. (Only read is appropriate for each frame)

3. Conclusion

Use grab to skip unnecessary frames and read to get the required frames.

It was possible to save 36% of the time.

4. Verification

Verification target: 　Luis Fonsi - Despacito ft. Daddy Yankee 　720p 30fps Video time 4 minutes 41 seconds

When using OpenCV :: VideoCapture to capture frames every second

1. set + read
2. grab + read

Measure the processing time of 2 patterns.

Verification environment Windows10 64bit Python 3.6.9 OpenCV 4.1.1 numpy 1.16.5

1. set + read Result 10.131 seconds

set+read

import cv2


movie_path = "./movie/Luis Fonsi - Despacito ft. Daddy Yankee.mp4"
video = cv2.VideoCapture(movie_path)

#Get the number of frames
frame_count = int(video.get(7))

#frame rate(The time unit of one frame is milliseconds)Get
frame_rate = int(video.get(5))

#Every n seconds
n = 1

#Total number of acquired frames
frame_all = int((frame_count / frame_rate) / n)

for i in range(frame_all):  #Turn by the total number of acquired frames
    frame_set = frame_rate * n * i
    if frame_set > frame_count:
        break

    video.set(1, frame_set)
    ret, work_frame = video.read()
    if ret is False:
        break

    #Here is the processing for the frame

video.release()

set+read result

         569 function calls in 10.131 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      281    8.791    0.031    8.791    0.031 {method 'set' of 'cv2.VideoCapture' objects}
      281    1.248    0.004    1.248    0.004 {method 'read' of 'cv2.VideoCapture' objects}
        1    0.087    0.087   10.131   10.131 time_check.py:4(test)
        1    0.006    0.006    0.006    0.006 {method 'release' of 'cv2.VideoCapture' objects}
        1    0.000    0.000   10.131   10.131 <string>:1(<module>)
        1    0.000    0.000   10.131   10.131 {built-in method builtins.exec}
        2    0.000    0.000    0.000    0.000 {method 'get' of 'cv2.VideoCapture' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

2. grab + read

Result 6.491 seconds

grab+read

import cv2


movie_path = "./movie/Luis Fonsi - Despacito ft. Daddy Yankee.mp4"
video = cv2.VideoCapture(movie_path)

#Get the number of frames
frame_count = int(video.get(7))

#frame rate(The time unit of one frame is milliseconds)Get
frame_rate = int(video.get(5))

#Every n seconds
n = 1

#Read interval setting
read_interval = int((frame_rate * n) - 1)

for i in range(frame_count):  #Turn for a few frames
    ret = video.grab()
    if ret is False:
        break

    if i % read_interval is 0:
        ret, work_frame = video.read()
        if ret is False:
            break

        #Here is the processing for the frame

video.release()

grab+read result

         8437 function calls in 6.491 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     8149    5.154    0.001    5.154    0.001 {method 'grab' of 'cv2.VideoCapture' objects}
      281    1.232    0.004    1.232    0.004 {method 'read' of 'cv2.VideoCapture' objects}
        1    0.098    0.098    6.490    6.490 time_check_set.py:4(test)
        1    0.006    0.006    0.006    0.006 {method 'release' of 'cv2.VideoCapture' objects}
        1    0.000    0.000    6.490    6.490 <string>:1(<module>)
        1    0.000    0.000    6.491    6.491 {built-in method builtins.exec}
        2    0.000    0.000    0.000    0.000 {method 'get' of 'cv2.VideoCapture' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

5. Conclusion

Thank you for watching until the end. This process is about 3 seconds shorter than a 4 minute video, so I don't think it's too big.

・ Analyze long videos ・ Scene where specifications such as rental server are limited

Then it seems that it can be more effective.