Introduction

This time, I am modifying the previous program, but at that time I am also changing the variable names etc. in detail.

Functionalize drawing Japanese fonts with OpenCV
Make the function of drawing Japanese fonts with OpenCV general purpose
Handling transparent images with OpenCV-Making sprites dance-
Rotate sprites with OpenCV
Rotate sprites with OpenCV # 2 ~ Mastering cv2.warpAffine () ~
Rotate sprites with OpenCV # 3 ~ Calculate by yourself without leaving it to others ~ ← Now here

Aiming figure

About the sprite-like superposition function of the old hobby personal computer 　　putSprite(back, front4, pos, angle=0, home=(0,0)) And. The expression is almost the same as the previous articles, but the meaning of the argument is changed.

-* Back * Background image. RGB3 channel. -* front4 * Foreground image you want to overlay. 4 channels of RGBA. Semi-transparent is not supported. Translucent doesn't really interest me. -* pos * The coordinates of the origin specified by *** home * are used instead of the upper left of the foreground image. -* Angle * Rotation angle. The unit is degrees and the default value is 0. -* Home * The origin of sprite display and rotation. The default value is the upper left, that is, (0,0).

I think this one is easier to use.

Basic program

It's a hassle to animate every time, so I made something like this.

`sample.py`


import cv2
import numpy as np

def makeSampleImg(img4):
    h, w = img4.shape[:2]
    cv2.rectangle(img4, (0,0), (w-1,h-1), (0,0,255,255), 1)
    return img4

def putSprite(img_back, img_front, pos, angle=0, home=(0,0)):
    #Implement in various ways and choose the best one.
    pass

def main():
    img_front = cv2.imread("uchuhikoushi.png ", -1)
    img_front = makeSampleImg(img_front)
    img_back = cv2.imread("space.jpg ", -1)
    pos = (100,80)
    home = (140,60)
    angle = 30

    #This is the main. Change the function name as needed
    img = putSprite(img_back.copy(), img_front, pos, angle, home)

    cv2.circle(img, pos, 5, (0,0,255), -1)  #Same coordinates(pos)Draw a circle
    cv2.imshow("rotation", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

The result is this. Should be.

result

Find the smallest circumscribed quadrangle

In "Rotate sprites with OpenCV", I couldn't find the upper left coordinates of the rotated image, but I managed to calculate it after that. As I knew, it was high school level math.

`putSprite_calc`


def putSprite_calc(back, front4, pos, angle=0, home=(0,0)):
    fh, fw = front4.shape[:2]
    bh, bw = back.shape[:2]
    x, y = pos
    xc, yc = home[0] - fw/2, home[1] - fh/2             #Change home from the upper left reference to the image center reference
    a = np.radians(angle)
    cos , sin = np.cos(a), np.sin(a)                    #This trigonometric function appears many times, so make it a variable
    w_rot = int(fw * abs(cos) + fh * abs(sin))
    h_rot = int(fw * abs(sin) + fh * abs(cos))
    M = cv2.getRotationMatrix2D((fw/2,fh/2), angle, 1)  #Rotate in the center of the image
    M[0][2] += w_rot/2 - fw/2
    M[1][2] += h_rot/2 - fh/2
    imgRot = cv2.warpAffine(front4, M, (w_rot,h_rot))   #Circumscribed rectangle containing rotated image

    #Do nothing if the entire circumscribed rectangle is outside the background image
    xc_rot = xc * cos + yc * sin                        #Amount of movement when rotated in the center of the image
    yc_rot = -xc * sin + yc * cos
    x0 = int(x - xc_rot - w_rot / 2)                    #Upper left coordinates of the circumscribed quadrangle
    y0 = int(y - yc_rot - h_rot / 2)
    if not ((-w_rot < x0 < bw) and (-h_rot < y0 < bh)) :
        return back
    
    #Get only the background image of the circumscribed quadrangle
    x1, y1 = max(x0,  0), max(y0,  0)
    x2, y2 = min(x0 + w_rot, bw), min(y0 + h_rot, bh)
    imgRot = imgRot[y1-y0:y2-y0, x1-x0:x2-x0]

    #Combine the circumscribed rectangle and the background with the mask method
    result = back.copy()
    front = imgRot[:, :, :3]
    mask1 = imgRot[:, :, 3]
    mask = 255 - cv2.merge((mask1, mask1, mask1))
    roi = result[y1:y2, x1:x2]
    tmp = cv2.bitwise_and(roi, mask)
    tmp = cv2.bitwise_or(tmp, front)
    result[y1:y2, x1:x2] = tmp
    return result

imgRot

Modification of square ROI

In "Rotate sprites with OpenCV", I ended up using a uselessly large square with known dimensional relationships. The function at this time is modified to this specification.

`putSprite_mask2 break`


def putSprite_mask2(back, front4, pos, angle=0, home=(0,0)):
    fh, fw = front4.shape[:2]
    bh, bw = back.shape[:2]
    x, y = pos
    xc, yc = home

    #Find the maximum distance between the center of rotation and the four corners
    pts = np.array([(0,0), (fw,0), (fw,fh), (0,fh)])
    ctr = np.array([(xc,yc)])
    r = int(np.sqrt(max(np.sum((pts-ctr)**2, axis=1))))

    #Square with rotated image
    M = cv2.getRotationMatrix2D((xc,yc), angle, 1)      #Rotate at home
    M[0][2] += r - xc
    M[1][2] += r - yc
    imgRot = cv2.warpAffine(front4, M, (2*r,2*r))       #Square with rotated image

    #Do nothing if the entire rectangle is outside the background image
    x0, y0 = x-r, y-r
    if not ((-2*r < x0 < bw) and (-2*r < y0 < bh)) :
        return back    

    #Get only the background image of the rectangle
    x1, y1 = max(x0,  0), max(y0,  0)
    x2, y2 = min(x0+2*r, bw), min(y0+2*r, bh)
    imgRot = imgRot[y1-y0:y2-y0, x1-x0:x2-x0]

    #Combine the circumscribed rectangle and the background with the mask method
    result = back.copy()
    front = imgRot[:, :, :3]
    mask1 = imgRot[:, :, 3]
    mask = 255 - cv2.merge((mask1, mask1, mask1))
    roi = result[y1:y2, x1:x2]
    tmp = cv2.bitwise_and(roi, mask)
    tmp = cv2.bitwise_or(tmp, front)
    result[y1:y2, x1:x2] = tmp
    return result

imgRot

Minimize square ROI

When I was wondering if I could get the smallest circumscribed quadrangle from this uselessly large square programmatically rather than mathematically, I found the article I was looking for.

Understanding how to get information about the rectangle surrounding the nonzero region of a two-dimensional array of numpy (Kei Minagawa's Blog)

I was going to take care of this if I couldn't solve the problem of figures to the end.

imgRot	Minimize

Modification of sprite function by cv2.warpAffine ()

"Rotate sprites with OpenCV # 2 ~ Mastering cv2.warpAffine () ~" Let's also modify the program. Since the image size during rotation is equal to that of the background image, the execution speed will slow down as soon as you try to place many small sprites on the large background image.

imgRot

`putSprite_Affine2`


def putSprite_Affine2(back, front4, pos, angle=0, home=(0,0)):
    x, y = pos
    xc, yc = home
    front3 = front4[:, :, :3]
    mask1 =  front4[:, :, 3]
    mask3 = 255- cv2.merge((mask1, mask1, mask1))
    bh, bw = back.shape[:2]

    M = cv2.getRotationMatrix2D(home, angle, 1)
    M[0][2] += x - xc  #The only change point is here where the definition of pos was changed.
    M[1][2] += y - yc  #No need for extra calculations cv2.warpAffine()Strengths.
    front_rot = cv2.warpAffine(front3, M, (bw,bh))
    mask_rot = cv2.warpAffine(mask3, M, (bw,bh), borderValue=(255,255,255))
    tmp = cv2.bitwise_and(back, mask_rot)
    result = cv2.bitwise_or(tmp, front_rot)
    return result

Execution speed comparison

Add a rotation element to the comparison program created in Handling transparent images with OpenCV-Making sprites dance- and execute it.

`rot_test.py`


import cv2
import numpy as np
import time

# def makeSampleImg(img4)Is unnecessary

def putSprite_calc(back, front4, pos, angle=0, home=(0,0)):
    #The ones listed above

def putSprite_mask2(back, front4, pos, angle=0, home=(0,0)):
    #The ones listed above

def putSprite_Affine2(back, front4, pos, angle=0, home=(0,0)):
    #The ones listed above

def main(func):
    filename_back = "space.jpg "
    filename_front = "uchuhikoushi.png "
    img_back = cv2.imread(filename_back)
    img_front = cv2.imread(filename_front, -1)
    bh, bw = img_back.shape[:2]
    xc, yc = bw//2, bh//2
    rx, ry = bw*0.3, bh*0.4
    home = (140,60)
    cv2.putText(img_back, func, (20,bh-20), cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,255))

    ###Start from here to measure the time
    start_time = time.time()

    for angle in range(-180, 180, 10):
        back = img_back.copy()
        x = int(xc + rx * np.cos(np.radians(angle)))
        y = int(yc + ry * np.sin(np.radians(angle)))
        img = eval(func)(img_back, img_front, (x,y), angle=angle, home=home)

        #This can be enabled or disabled as needed
        #cv2.imshow(func, img)
        #cv2.waitKey(1)

    elasped_time = time.time() - start_time
    ###So far

    print (f"{func} : {elasped_time} sec")    
    cv2.destroyAllWindows()


if __name__ == "__main__":
    funcs = ["putSprite_calc",
             "putSprite_mask2",
             "putSprite_Affine2" ]
    for func in funcs:
        for i in range(10):
            main(func)

The animation created has various elements added for the sake of clarity, but it looks like the one below.

result

The time it takes to process this is in my environment.

`python`


putSprite_calc    : 0.12500691413879395 sec
putSprite_mask2   : 0.27501583099365234 sec
putSprite_Affine2 : 0.5620322227478027 sec

As I knew, the smaller the ROI area, the faster the execution speed. The size of ROI is directly linked to the amount of calculation of vertical x horizontal x RGB3 channels, so just because OpenCV does a good job, if you use a wastefully large ROI, it will quickly slow down.

At the end

Now that I've made it so far, I want to make a game. I have to study deep learning as well. If it is forcibly related to deep learning, it will be possible to reduce the amount of calculation by reducing the size in a good way even when turning deep learning.

[PYTHON] Rotate sprites with OpenCV # 3 ~ Calculate by yourself without leaving it to others ~