Details of cv2.warpAffine ()

There was an interesting non-essential argument of cv2.warpAffine () that I omitted last time. Reference article: Draw another image on the image with OpenCV

--src Original image. Required. --M 2 * 3 transformation matrix. Required. --dsize Specify the output image size with a tuple of (width, height). Required. --dst background (outside the original image) image. The size should be the same as dsize. --flags Image complement method. The default value is cv2.INTER_LINEAR. Others such as cv2.INTER_NEAREST. --BorderMode Background (outside the original image) processing method. The default value is cv2.BORDER_CONSTANT. --BorderValue The color when borderMode = cv2.BORDER_CONSTANT. The default value is 0. See below.

Small-process the original image in advance

To make the result of image processing easy to understand, create a small function. Please note that only the minimum part is described.

python

def makeSampleImg(img4):   #Bring an RGBA 4-channel image
    h, w = img.shape[:2]
    #A as well as coloring the surroundings=The miso is to make it 255 (opaque)
    cv2.rectangle(img, (0,0), (w-1,h-1), (0,0,255,255), 1)
    return img

img_origin = cv2.imread(filename, -1) #RGBA image
img4 = makeSampleImg(img_origin)
img3 = img4[:, :, :3]  #Eliminate the A element and make an RGB image

The original image	Small processing	3 channels

Fill the background with a single color

Make the background a single color with borderMode = cv2.BORDER_CONSTANT. Since this is the default value of borderMode, there is no need to describe it. I'll try some.

Specify background color

M = cv2.getRotationMatrix2D((w/2,h/2), 30, 1)
img_rot31 = cv2.warpAffine(img3, M, (w, h), borderValue=(255,255,0))     #Foreground RGB, background RGB
img_rot32 = cv2.warpAffine(img3, M, (w, h), borderValue=(255,255,0,0))   #Foreground RGB, background RGBA (A component = 0)
img_rot33 = cv2.warpAffine(img3, M, (w, h))                              #Foreground RGB, no background specified
img_rot41 = cv2.warpAffine(img4, M, (w, h), borderValue=(255,255,0))     #Foreground RGBA, background RGB
img_rot42 = cv2.warpAffine(img4, M, (w, h), borderValue=(255,255,0,0))   #Foreground RGBA, background RGBA (A component = 0)
img_rot43 = cv2.warpAffine(img4, M, (w, h))                              #Foreground RGBA, no background specified
img_rot44 = cv2.warpAffine(img4, M, (w, h), borderValue=(255,255,0,255)) #Foreground RGBA, background RGBA (with A component)

** If the foreground is an RGB image **, even if you specify 4 channels of RGBA as the background color, the A component is ignored (img_rot32). It should be understood that if borderValue is not specified, the background will be black (img_rot33, or rather the default operation), but the background color will be (0,0,0).

img_rot31	img_rot32	img_rot33

** If the foreground is an RGBA image **, the output result will be 4 RGBA channels even if 3 RGB channels are specified as the background color. The A component given at this time is 0. Since A is opacity, not transparency, this is 0, which means that even if a BGR value that was not RGB ... is defined, it will be transparent as a result (img_rot41). Even if borderValue is not specified, the background becomes transparent (img_rot43), but it is easy to understand if you think that the background color will be (0,0,0,0). Of course, if you set a value other than 0 to the A component, it will be colored properly (img_rot44).

img_rot41	img_rot42	img_rot43	img_rot44

Set an image as the background

If you set borderMode = cv2.BORDER_TRANSPARENT, you can specify the background image with dst. As is often the case with OpenCV, cv2.warpAffine () will process the background image specified by dst. If you need to keep the original image, you need to set dst = back.copy ().

Since it is TRANSPARENT, it becomes transparent, so I expected that it would be a transparent background if I did not specify dst, but it was not so easy. Oh! Somehow I want to eat salmon roe.

Specify background image

back = cv2.imread(back_name)  #RGB image of the same size as the foreground image
back4 = cv2.cvtColor(back, cv2.COLOR_BGR2BGRA) #Make an RGBA image A component is 255 instead of 0

M = cv2.getRotationMatrix2D((w/2,h/2), 30, 1)
img_rot35 = cv2.warpAffine(img3, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back.copy())   #Foreground RGB, background RGB
img_rot36 = cv2.warpAffine(img3, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back4.copy())  #Foreground RGB, background RGBA
img_rot37 = cv2.warpAffine(img3, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT)                    #Foreground RGB, no background specified
img_rot45 = cv2.warpAffine(img4, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back.copy())   #Foreground RGBA, background RGB
img_rot46 = cv2.warpAffine(img4, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back4.copy())  #Foreground RGBA, background RGBA
img_rot47 = cv2.warpAffine(img4, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT)                    #Foreground RGBA, no background specified

Both the foreground and background behaved as expected with the RGB image (img_rot35).

If I didn't specify a background image in the RGB foreground, the result changed every time I ran it (img_rot37). numpy has a function callednumpy.empty ()that creates an uninitialized array. The background image (or rather the numpy array) is probably created with the same specifications here as well. I'm not sure why img_rot36, which specified the RGBA background in the RGB foreground, turned into a black background with dust.

img_rot35	img_rot36	img_rot37 part 1	img_rot37 part 2

The RGBA image (img_rot46) for both the foreground and background is as expected, but it is also a disappointing result. I was happy if the background was displayed in the transparent part of the foreground, but ** affine transformation is not that kind of thing **, so it can't be helped. The background of img_rot45, which specified an RGB background for the RGBA foreground, and img_rot47, which did not specify a background image for the RGBA foreground, became transparent. In both cases, it seems that 0 was added to the A element.

In short, the mundane conclusion is that it is wise not to use it unexpectedly.

img_rot45	img_rot46	img_rot47

Use for sprites

As mentioned above, referring to the article of my predecessor, I thought about pasting an RGBA image including transparency on the image, but it was impossible to conclude. However, the specifications that do not cause an error even if it extends beyond the background image are attractive. Therefore, I decided to combine the mask method that I have done so far with the affine transformation. Then ... it's done, super easy. No circumscribed quadrangle, no ROI, no hassle.

Source

import cv2

def makeSampleImg(img4):
    h, w = img4.shape[:2]
    cv2.rectangle(img4, (0,0), (w-1,h-1), (0,0,255,255), 1)
    return img4

def putSprite_Affine(back, front4, pos, angle=0, center=(0,0)):
    x, y = pos
    front3 = front4[:, :, :3]
    mask1 =  front4[:, :, 3]
    mask3 = 255- cv2.merge((mask1, mask1, mask1))
    bh, bw = back.shape[:2]

    M = cv2.getRotationMatrix2D(center, angle, 1)
    M[0][2] += x
    M[1][2] += y
    front_rot = cv2.warpAffine(front3, M, (bw,bh))
    mask_rot = cv2.warpAffine(mask3, M, (bw,bh), borderValue=(255,255,255))
    tmp = cv2.bitwise_and(back, mask_rot)
    result = cv2.bitwise_or(tmp, front_rot)
    return result

if __name__ == "__main__":
    filename_front = "uchuhikoushi.png "
    filename_back = "space.jpg "
    img_front = cv2.imread(filename_front, -1)
    img_front = makeSampleImg(img_front)  #Add a frame to the RGBA image (not required)
    img_back = cv2.imread(filename_back)

    pos = [(0, 50), (300,200), (400,400), (500,-50), (-100,1000)] #Upper left coordinate to put the image
    xc, yc = 140, 60  #Center of rotation of the foreground image
    angle = 0

    while True:
        img = img_back.copy()
        for x,y in pos:
            img = putSprite_Affine(img, img_front, (x,y), angle, (xc,yc))

            #Make sure it is portrayed correctly (not required)
            cv2.circle(img, (x,y), 5, (0,255,0), -1)       #Mark in the upper left of the foreground image
            cv2.circle(img, (x+xc,y+yc), 5, (0,0,255), -1) #Mark at the center of rotation
            cv2.putText(img, f"angle={angle}", (10,440), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 2)

        cv2.imshow("putSprite_Affine", img)

        key = cv2.waitKey(1) & 0xFF
        if key == ord("q"):
            break
        angle = (angle + 30) % 360

    cv2.destroyAllWindows()

Speaking of drawbacks, the mask image and RGB image are affine-transformed for the entire background image, so the amount of calculation is uselessly large ("[Overhead](https://ja.wikipedia.org/wiki/%E3%82%]" AA% E3% 83% BC% E3% 83% 90% E3% 83% BC% E3% 83% 98% E3% 83% 83% E3% 83% 89) seems to be large. "

Front in the middle of drawing_rot
Mask in the middle of drawing_rot
result
Here, the previous anime GIF is posted, but in reality, the character with the red frame rotates.

Now let's see how this affects execution speed.

At the end

It's not over yet.