Try loading the image in a separate thread (OpenCV-Python)

Recently I found out about a library called trio.

[Python + Trio asynchronous coding pattern] (https://qiita.com/yura/items/689d065aba00fe14fbba)

Please refer to the.

If you write an image processing / image recognition program, you will want to load the next image during the image processing / recognition time. So I decided to write a multithreaded program in Python.

1. Write a serialized version.
2. Review the multithreaded framework in that language.
3. Think about which process and which process are independent and how multithreading is possible.
4. Implement the multithreaded version

Write a sequential version.

I wrote a sequential processing version as described later. I want to load the next image during this detection process. For operation, prepare an input image to detect people, pat = r"*.png " Please rewrite the part of.

Sequential processing version

detectLoop.py

# -*- coding: utf-8 -*-
# pylint: disable-msg=C0103
import glob
import cv2
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())

def detect(img):
    found = hog.detectMultiScale(img, winStride=(8, 8), padding=(32, 32), scale=1.05)
    return found

def draw_detections(img, rects, thickness=1):
    for x, y, w, h in rects:
        # the HOG detector returns slightly larger rectangles than the real objects.
        # so we slightly shrink the rectangles to get a nicer output.
        pad_w, pad_h = int(0.15*w), int(0.05*h)
        cv2.rectangle(img, (x+pad_w, y+pad_h), (x+w-pad_w, y+h-pad_h), (0, 255, 0), thickness)

if __name__ == '__main__':
    pat = r"*.png "
    names = glob.glob(pat)[:100]

    e1 = cv2.getTickCount()

    imgname = names[0]
    img = cv2.imread(imgname)

    for i in range(1, len(names), 1):
        found, scores = detect(img)
        imgnameNext = names[i]
        imgNext = cv2.imread(imgnameNext)
        if 0:
            print i, imgnameNext, imgNext.shape
            print found
        oldImg, imgname, img = img, imgnameNext, imgNext

        draw_detections(oldImg, found)
        cv2.imshow("detected", oldImg)
        cv2.waitKey(1)

    e2 = cv2.getTickCount()
    timeDiff = (e2 - e1)/ cv2.getTickFrequency()
    print timeDiff

Review the multithreaded framework in that language

"There are two ways to handle threads in the threading module. Create a subclass of threading.Thread. Create an instance of threading.Thread. " However, let's choose a method to create an instance directly.

First, let's look at a multithreaded example of processing that does not return a result from a function.

Then, it became clear that the next part was the main point.

\ # Create an instance of the thread. t = threading.Thread (target = function, args = tuple of function arguments) t.start() Other processing t.join()

In this example, the result of the function cannot be returned, so in order to return the result between threads, Queue is used to pass data between threads.

def function name(queue,argument):
Function processing
    queue.put(Return value)

t = threading.Thread(target=Function name, args=(queue,argument)) 
t.start()
#Write the process
t.join()
Function return value= queue.get()

Think about how multithreading is possible

There are two ways to consider whether to use a separate thread for image capture or person detection processing. Here, I tried to make the person detection process a separate thread. By writing like this, loading the next image and processing the loaded image It seems that it can be operated in parallel.

Implement a multithreaded version

Multithreaded version

detectLoop_thread.py

# -*- coding: utf-8 -*-
# pylint: disable-msg=C0103
import threading
import Queue
import glob
import cv2

global hog
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())

def detect2(queue, img):
    global hog
    found = hog.detectMultiScale(img, winStride=(8, 8), padding=(32, 32), scale=1.05)
    queue.put(found)

def draw_detections(img, rects, thickness=1):
    for x, y, w, h in rects:
        # the HOG detector returns slightly larger rectangles than the real objects.
        # so we slightly shrink the rectangles to get a nicer output.
        pad_w, pad_h = int(0.15*w), int(0.05*h)
        cv2.rectangle(img, (x+pad_w, y+pad_h), (x+w-pad_w, y+h-pad_h), (0, 255, 0), thickness)

if __name__ == '__main__':
    pat = r"*.png "
    names = glob.glob(pat)[:100]

    e1 = cv2.getTickCount()
    queue = Queue.Queue()
    imgname = names[0]
    img = cv2.imread(imgname)

    for i in range(1, len(names), 1):
        t = threading.Thread(target=detect2, args=(queue, img, ))
        t.start()
        imgnameNext = names[i]
        imgNext = cv2.imread(imgnameNext)
        t.join()
        found, scores = queue.get()
        if 0:
            print i, imgnameNext, imgNext.shape
            print found
        oldImg, imgname, img = img, imgNext, imgNext
        draw_detections(oldImg, found)
        cv2.imshow("detected", oldImg)
        cv2.waitKey(1)

    e2 = cv2.getTickCount()
    timeDiff = (e2 - e1)/ cv2.getTickFrequency()
    print timeDiff

As for the operation result, the processing time is reduced by about 0.8 seconds when reading 100 png files. (In the example, it can be seen that it is necessary to reduce the time required for the person detection process itself.)

Sequential processing version seconds 29.5429292224 Multithreaded version of seconds 28.7976980869

# -*- coding: utf-8 -*-
# pylint: disable-msg=C0103
import threading
import Queue
import glob
import cv2

global hog
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())

def imread2(queue, imgnameNext):
    imgNext = cv2.imread(imgnameNext)
    queue.put(imgNext)

def draw_detections(img, rects, thickness=1):
    for x, y, w, h in rects:
        # the HOG detector returns slightly larger rectangles than the real objects.
        # so we slightly shrink the rectangles to get a nicer output.
        pad_w, pad_h = int(0.15*w), int(0.05*h)
        cv2.rectangle(img, (x+pad_w, y+pad_h), (x+w-pad_w, y+h-pad_h), (0, 255, 0), thickness)

if __name__ == '__main__':
    pat = r"*.png "
    names = glob.glob(pat)[:100]

    e1 = cv2.getTickCount()
    queue = Queue.Queue()
    imgname = names[0]
    img = cv2.imread(imgname)

    for i in range(1, len(names), 1):
        imgnameNext = names[i]
        t = threading.Thread(target=imread2, args=(queue, imgnameNext, ))
        t.start()
        found, scores = hog.detectMultiScale(img, winStride=(8, 8), padding=(32, 32), scale=1.05)
        t.join()
        imgNext = queue.get()
        
        if 0:
            print i, imgnameNext, imgNext.shape
            print found
        oldImg, imgname, img = img, imgNext, imgNext
        draw_detections(oldImg, found)
        cv2.imshow("detected", oldImg)
        cv2.waitKey(1)

    e2 = cv2.getTickCount()
    timeDiff = (e2 - e1)/ cv2.getTickFrequency()
    print timeDiff