[PYTHON] Rotate sprites with OpenCV

Introduction

This is a continuation of Handling transparent images with OpenCV-Making sprites dance-.

It's exciting to rotate sprites. I remember Namco's SYSTEM II board. Assault or ordyne. I bought Namco Museum VOL.4 before I bought the PlayStation itself.

Rotate images with OpenCV

In OpenCV, you can rotate the image in 90 degree increments with cv2.rotate (), but there is no function to rotate it at any angle. There are more advanced functions, so use them.

Studying affine transformation

An affine transformation is a mapping that combines primary transformation such as rotation and scaling with translation. In short

\begin{pmatrix}
x' \\
y'
\end{pmatrix}
=
\begin{pmatrix}
a & b \\
d & e
\end{pmatrix}
\begin{pmatrix}
x \\
y
\end{pmatrix}
+
\begin{pmatrix}
c \\
f
\end{pmatrix}

It is a conversion. Because it is difficult to program in this form

\begin{pmatrix}
x' \\
y' \\
1
\end{pmatrix}
=
\begin{pmatrix}
a & b & c \\
d & e & f \\
0 & 0 & 1
\end{pmatrix}
\begin{pmatrix}
x \\
y \\
1
\end{pmatrix}

I will express it as. The bottom is somewhat unpleasant, but there is only an identity of 1 = 1.

this

\begin{pmatrix}
a & b & c \\
d & e & f \\
\end{pmatrix}

Is defined and applied to a function called cv2.warpAffine (). However, when rotating the image, bother

\begin{pmatrix}
cosθ & -sinθ & 0 \\
sinθ & cosθ & 0 \\
\end{pmatrix}

There is no need to calculate. You can use the matrix obtained by cv2.getRotationMatrix2D ().

Create a rotation matrix cv.getRotationMatrix2D (center, angle, scale)

--center Rotation center. --angle Rotation angle. Counterclockwise, ** units are degrees ** instead of radians. --Scale Magnification of scaling.

Let's add the translation by yourself.

Convert image to affine cv.warpAffine (src, M, dsize)

--src Original image. --M 2 * 3 transformation matrix. --dsize Specify the output image size with a tuple of (width, height). --Other non-essential arguments are omitted.

Practice

I wonder what the transformation matrix is like.

Source 1


import cv2
angle = 30  # degrees
M = cv2.getRotationMatrix2D((0,0), angle, 1)
print (M)
print (M.shape)
print (M.dtype)

Result 1


[[ 0.8660254  0.5        0.       ]
 [-0.5        0.8660254  0.       ]]
(2, 3)
float64

cos30 degrees = √3 / 2 = 0.8660254 ... So that's exactly what the rotation matrix is ...? The minus position is different, right? Apparently, unlike the xy plane used in mathematics, the calculation corresponds to the coordinate system where the bottom is positive.

So, let's actually convert the image using this.

Source 2


import cv2

filename = "hoge.png "
img = cv2.imread(filename)
h, w = img.shape[:2]
center = (140,60)
angle = 0

while True:
    angle = (angle + 10) % 360
    M = cv2.getRotationMatrix2D(center, angle, 1)
    img_rot = cv2.warpAffine(img, M, (w, h))
    cv2.imshow(filename, img_rot)
    key = cv2.waitKey(100) & 0xFF
    if key == ord("q"):
        break
    
cv2.destroyAllWindows()

The result is this.

The original image result
uchuhikoushi_point.png uchuhikoushi_anim_0.gif

I understand the rotation angle and the center of rotation, but I want to say, "Wait a minute, the image is sticking out in the first place."

Avoid overhang

How to avoid sticking out? I'll calculate what to do.

rot_cad_2.png

When a rectangle (red) whose two sides are w and h is tilted by an angle a, the size of the circumscribed rectangle (blue) is

python


rot_w = w*cos(a) + h*sin(a)
rot_h = w*sin(a) + h*cos(a)

Will be. Since sine and cosine become negative when the degree deviates from 0 to 90 degrees, each term must be an absolute value to be exact. Oh, I finally understand. This formula appeared in the article of the ancestors described later, and I was wondering what it was different from the rotation matrix.

So, you can translate the position so that it fits in this blue rectangle. Since (0,0) is the center of rotation, we think based on the center of the image.

Source 3


import cv2
import numpy as np

filename = "hoge.png "
img = cv2.imread(filename)
h, w = img.shape[:2]
angle = 0

while True:
    angle = (angle + 10) % 360
    a = np.radians(angle)
    w_rot = int(np.round(w*abs(np.cos(a)) + h*abs(np.sin(a))))
    h_rot = int(np.round(w*abs(np.sin(a)) + h*abs(np.cos(a))))

    M = cv2.getRotationMatrix2D((w/2,h/2), angle, 1)
    M[0][2] += -w/2 + w_rot/2
    M[1][2] += -h/2 + h_rot/2

    img_rot  = cv2.warpAffine(img, M, (w_rot,h_rot))
    cv2.imshow(filename, img_rot)
    key = cv2.waitKey(100) & 0xFF
    if key == ord("q"):
        break
    
cv2.destroyAllWindows()

The result is this. The image no longer sticks out of the window, but it can't be helped because the animation is based on the constraint that the window position (upper left position of the image) is common.

uchuhikoushi_anim_1.gif

Specify the center of rotation

In order to specify the center and rotate it so that the created image does not go wild, the size of the canvas on which the rotated image is drawn should be determined in advance. Finding the size of the campus is not difficult. Of the distance between the center of rotation and the four corners, the largest one corresponds to the radius. The size of the canvas is twice that. rot_cad_3.png

Probably in good condition, the following source expresses the formula for finding the radius in just one line. Now it's hard to understand what kind of calculation I'm doing. It's not good to run very skillfully.

Source 4


import cv2
import numpy as np

filename = "hoge.png "
img = cv2.imread(filename)
h, w = img.shape[:2]
xc, yc = 140, 60 #Rotation center
angle = 0

#Find the maximum value of the distance between the center of rotation and the four corners
pts = np.array([(0,0), (w,0), (w,h), (0,h)])
ctr = np.array([(xc,yc)])
r = np.sqrt(max(np.sum((pts-ctr)**2, axis=1)))
winH, winW = int(2*r), int(2*r)

while True:
    angle = (angle + 10) % 360
    M = cv2.getRotationMatrix2D((xc,yc), angle, 1)
    M[0][2] += r - xc
    M[1][2] += r - yc
    
    imgRot = cv2.warpAffine(img, M, (winW,winH))
    cv2.imshow("", imgRot)
    key = cv2.waitKey(100) & 0xFF
    if key == ord("q"):
        break

cv2.destroyAllWindows()

The result is this. uchuhikoushi_anim_2.gif

Combine the rotated image with the background image

Add the above processing to the sprite function of previous. It might have been better to round up with math.ceil () instead of ʻint () `.

Source 5


import cv2
import numpy as np

def putSprite_mask2(back, front4, pos, angle=0, center=(0,0)):
    x, y = pos
    xc, yc = center
    fh, fw = front4.shape[:2]
    bh, bw = back.shape[:2]

    #Find the maximum value of the distance between the center of rotation and the four corners
    pts = np.array([(0,0), (fw,0), (fw,fh), (0,fh)])
    ctr = np.array([(xc,yc)])
    r = int(np.sqrt(max(np.sum((pts-ctr)**2, axis=1))))

    #Rotate
    M = cv2.getRotationMatrix2D((xc,yc), angle, 1)
    M[0][2] += r - xc
    M[1][2] += r - yc
    imgRot = cv2.warpAffine(front4, M, (2*r,2*r))  #Circumscribed rectangle containing rotated image

    #Do nothing if the entire circumscribed rectangle is outside the background image
    x0, y0 = x+xc-r, y+yc-r
    if not ((-2*r < x0 < bw) and (-2*r < y0 < bh)) :
        return back    

    #Get only the background image of the circumscribed quadrangle
    x1, y1 = max(x0,  0), max(y0,  0)
    x2, y2 = min(x0+2*r, bw), min(y0+2*r, bh)
    imgRot = imgRot[y1-y0:y2-y0, x1-x0:x2-x0]

    #Combine the circumscribed rectangle and the background with the mask method
    front_roi = imgRot[:, :, :3]
    mask1 = imgRot[:, :, 3]
    mask_roi = 255 - cv2.merge((mask1, mask1, mask1))
    roi = back[y1:y2, x1:x2]
    tmp = cv2.bitwise_and(roi, mask_roi)
    tmp = cv2.bitwise_or(tmp, front_roi)
    back[y1:y2, x1:x2] = tmp

    return back


if __name__ == "__main__":
    filename_back = "space.jpg "
    filename_front = "uchuhikoushi.png "
    img_back = cv2.imread(filename_back)
    img_front = cv2.imread(filename_front, -1)

    pos = [(0, 50), (300,200), (400,400), (500,-50), (-100,1000)] #Upper left coordinate to put the image
    xc, yc = 140, 60  #Center of rotation of the foreground image
    angle = 0

    while True:
        back = img_back.copy()
        for x,y in pos:
            img = putSprite_mask2(back, img_front, (x,y), angle, (xc,yc))

            #Make sure it is portrayed correctly (not required)
            cv2.circle(img, (x,y), 5, (0,255,0), -1)       #Mark in the upper left of the foreground image
            cv2.circle(img, (x+xc,y+yc), 5, (0,0,255), -1) #Mark at the center of rotation
            cv2.putText(img, f"angle={angle}", (10,440), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 2)

        cv2.imshow("putSprite_mask2", img)

        key = cv2.waitKey(0) & 0xFF
        if key == ord("q"):
            break
        angle = (angle + 30) % 360
        
    cv2.destroyAllWindows()

It's done. uchuhikoushi_anim_3.gif

At the end

Actually, I wanted to achieve the same behavior by minimizing the circumscribed quadrangle and finely controlling the upper left coordinates, but I could not solve the problems of figures and matrices, that is, high school mathematics level well. There wasn't. Eh? Why don't you line up in high school now? !!

Reference article

Creates and saves an image that is rotated around the center of the original image by the specified angle. How to prevent the protruding part from being cut off by rotating the image of opencv Completely understand affine transformation

Recommended Posts

Rotate sprites with OpenCV
Rotate sprites with OpenCV # 2 ~ Mastering cv2.warpAffine () ~
Detect stoop with OpenCV
Binarization with OpenCV / Python
Data Augmentation with openCV
Rotate sprites with OpenCV # 3 ~ Calculate by yourself without leaving it to others ~
Easy TopView with OpenCV
Stumble with homebrew opencv3
Face recognition with Python's OpenCV
"Apple processing" with OpenCV3 + Python3
Try edge detection with OpenCV
Image editing with python OpenCV
Camera capture with Python + OpenCV
[Python] Using OpenCV with Python (Basic)
Binarize photo data with OpenCV
Loop video loading with opencv
Face detection with Python + OpenCV
Get image features with OpenCV
Face recognition / cutting with OpenCV
Try OpenCV with Google Colaboratory
Cascade classifier creation with opencv
Using OpenCV with Python @Mac
Image recognition with Keras + OpenCV
Anime face detection with OpenCV
Shining life with Python and OpenCV
Real-time image processing basics with opencv
Draw unblurred sprites with cocos2d / pyglet
[Python] Using OpenCV with Python (Image Filtering)
Neural network with OpenCV 3 and Python 3
I tried trimming efficiently with OpenCV
[Python] Using OpenCV with Python (Image transformation)
[Python] Using OpenCV with Python (Edge Detection)
Erase certain colors with OpenCV + PySimpleGUI
Introducing OpenCV on Mac with homebrew
Easy Python + OpenCV programming with Canopy
Finding the simplest mistakes with OpenCV
[OpenCV] Personal identification with face photo
Try face recognition with python + OpenCV
Cut out face with Python + OpenCV
Face recognition with camera with opencv3 + python2.7
OpenCV feature detection with Google Colaboratory
Search for homeomorphic idioms with opencv
Load gif images with Python + OpenCV
Cat detection with OpenCV (model distribution)
Find image similarity with Python + OpenCV
Try blurring the image with opencv2
Use OpenCV with Python 3 in Window
Draw an illustration with Python + OpenCV
Track baseball balls with Python + OpenCV
Graph Based Segmentation with Python + OpenCV
Object recognition with openCV by traincascade
Draw shapes with OpenCV and PIL
I tried face recognition with OpenCV
Implemented inter-frame difference method with OpenCV
Draw arrows (vectors) with opencv / python
Basic study of OpenCV with Python
I want to detect objects with OpenCV
Make a video player with PySimpleGUI + OpenCV
Save video frame by frame with Python OpenCV
Try using the camera with Python's OpenCV
Capturing images with Pupil, python and OpenCV