[PYTHON] Make the function of drawing Japanese fonts in OpenCV general

Introduction

I received a comment in the previous article "Functionalizing Japanese fonts in OpenCV" (https://qiita.com/mo256man/items/82da5138eeacc420499d). I am very grateful. What's more, the place where I skipped is clearly seen. I have to do my best for this.

Improvement

About RGB and BGR

The target is a function that can describe Japanese in the same way as cv2.imshow (), so the specified color is the OpenCV sequence (BGR). In the previous article, when working with PIL in a function, I converted it to PIL color sequence (RGB). The image was also converted from BGR to RGB, Japanese was drawn with the PIL function, returned to BGR, and then converted back to OpenCV.

More elegant color order reversal

Regarding inversion of color order In the above, the non-elegant implementation method of taking out each element and changing the order was used, but it seems that slices can be used normally.

python

>>> (255, 0, 128)[::-1]
(128, 0, 255)

What? I should have tried this after a lot of trial and error.

Or rather, there is no need to reverse the color order

so. With the condition that you understand the relationship between BGR and RGB. If you do not convert the image to the correct color for PIL, draw the sequence that should be RGB as the text color with PIL as BGR, and return it to OpenCV as it is, it will be the correct color as OpenCV as a result. It's fast, isn't it? That is the purpose of the person who commented. When I submitted it to the school teacher as an assignment, "I told you that OpenCV and PIL have the opposite color arrangement. It seems that the answer is correct but the calculation formula in the middle is wrong." It's a technique that seems to be said, but programming requires this kind of idea. No, I also came up with it. However, I didn't incorporate it so much because I couldn't use the function that converts OpenCV and PIL that I made. Excuse me.

So let's compare the third function you posted with my second function.

python

#Original function
def cv2_putText_2(img, text, org, fontFace, fontScale, color):
    x, y = org
    b, g, r = color        #This sentence is unnecessary because the color remains BGR
    colorRGB = (r, g, b)   #This sentence is unnecessary because the color remains BGR. Don't even use slices
    imgPIL = cv2pil(img)   #The original function of converting the color and then converting it to PIL is simply converted to PIL.
    draw = ImageDraw.Draw(imgPIL)
    fontPIL = ImageFont.truetype(font = fontFace, size = fontScale)
    w, h = draw.textsize(text, font = fontPIL)
    draw.text(xy = (x,y-h), text = text, fill = colorRGB, font = fontPIL) #The color is the argument of the function as it is
    imgCV = pil2cv(imgPIL) #Without color reconversion
    return imgCV           #The return value is simply converted to OpenCV.

# ↓↓↓↓↓↓
#The function you posted
def cv2_putText_3(img, text, org, fontFace, fontScale, color):
    x, y = org
    imgPIL = Image.fromarray(img)
    draw = ImageDraw.Draw(imgPIL)
    fontPIL = ImageFont.truetype(font = fontFace, size = fontScale)
    w, h = draw.textsize(text, font = fontPIL)
    draw.text(xy = (x,y-h), text = text, fill = color, font = fontPIL)
    return np.array(imgPIL, dtype = np.uint8)

Well, it's nicely simple.

About ROI

ROI (Region of Interest) is translated as a region of interest and means an area of interest for processing in the entire image. [This article](https://qiita.com/mo256man/items/c570c645153cae122196#%E5%BF%9C%E7%94%A8%E4%BE%8B2%E9%A1%94%E6%A4%9C % E5% 87% BA% E3% 81% A8% E9% 80% A3% E5% 8B% 95% E3% 81% 97% E3% 81% 9F% E3% 83% A2% E3% 82% B6% E3 It is also used in% 82% A4% E3% 82% AF).

The next comment is that if you get the size of the text to be drawn in advance and PIL only the part of the original image where the characters are drawn, it will be much faster than PILing the entire original image. I don't think smart people are ... However, you need a ʻImageDraw.Draw object to use PIL's ʻImageDraw.textsize (). If you don't know the size, can you make a base monochrome image? No such thing. Even if the text is too small to be drawn, all you have to do is prepare the ʻImageDraw.Draw` object.

Let's read the 4th function posted based on that.

python

def cv2_putText_4(img, text, org, fontFace, fontScale, color):
    x, y = org
    fontPIL = ImageFont.truetype(font = fontFace, size = fontScale)

    #size(0,0)Create a solid image of and create its Draw object
    dummy_draw = ImageDraw.Draw(Image.new("RGB", (0,0)))

    #Gets the width and height of the rectangle when drawing the specified sentence with the specified font size
    w, h = dummy_draw.textsize(text, font = fontPIL)
    """
I'll add it here later
    """

    #Trim a part of the original image to the size you just got, and make only that part a PIL image
    imgPIL = Image.fromarray(img[y-h:y,x:x+w,:])

    #Create a Draw object for it
    draw = ImageDraw.Draw(imgPIL)

    #The standard is because it has been trimmed to draw the characters again.(0,0)
    draw.text(xy = (0,0), text = text, fill = color, font = fontPIL)

    #Reverse trimming Replace the relevant part of the original image with a character-depicted image
    img[y-h:y,x:x+w,:] = np.array(imgPIL, dtype = np.uint8)

    return img

Cheeks I see. However, due to a bug in ʻImageDraw.textsize ()`, if you use the obtained height as it is, the bottom will be cut off a little.

Then what would you do? We should expand such ROI to texto. Like this.

python

    b = int(0.1 * h) #Appropriately determine the height to be secured below the baseline,
    
    # imgPIL = Image.fromarray(img[y-h:y,x:x+w,:]) #To
    imgPIL = Image.fromarray(img[y-h:y+b,x:x+w,:]) #To

Correspondence to depiction outside the image

Since the OpenCV image is stored in the form of numpy.ndarray, you can extract a part by trimming by writing ʻimg [y: y + h, x: x + w] . This is one of the most moving facts I've learned about OpenCV, and I've written it over and over again. However, this has its drawbacks. Of course, since it is an array, it is not possible to specify outside the image. Of course, the cv2.putText ()` and graphic drawing functions provided by OpenCV are designed so that there is no problem even if they extend outside the original image, but my function will result in an error at this point. I have to trim it well.

I didn't write about PIL trimming in my article, but unlike OpenCV, I can specify outside the image range. https://note.nkmk.me/python-pillow-image-crop-trimming/ However, he showed me how to specify the ROI in the comment, and it seems that making the whole image PIL for cropping seems to go against the idea so far, and I was studying OpenCV in the first place. I don't want to use PIL trimming.

How about this way of thinking?

1. Obtain the text description size in advance and create a monochrome image of that size ①
2. Check if it sticks out when placed in the specified position on the original image Do nothing if the text area is completely out of the image
3. Replace all or part of the single color image with the original image ②
4. Draw text with PIL The brightness of R and the brightness of B are interchanged, but there is no problem if you do not show it ③
5. Return to OpenCV ④
6. Trim only the area where the original image was ⑤
7. Update the corresponding cell of the original image to the one with the characters drawn 6

①	②	③	④	⑤

⑥

Additional functions

Not only the lower left, but also the upper right and center coordinates can be specified for drawing.

The purpose from the beginning was to be able to handle Japanese fonts in the same way as cv2.putText (), but I wanted to add another function. The coordinates specified by * org * in cv2.putText () are at the bottom left of the character. As I wrote last time, I think this is difficult to use. Therefore, it is now possible to specify whether the coordinates specified by * org * should be the lower left as with cv2.putText (), the upper left as with PIL, or the center.

The image below shows that the same y-coordinate is specified in the original function (the y-coordinate of cv2.MARKER_STAR is the same), but it is drawn at different heights due to different settings. .. From the left, the same lower left standard as cv2.putText (), the upper right standard equivalent to PIL's ʻImageDraw.text ()`, and the center standard.

Tilt the letters to describe

Please let me go.

A function that incorporates the above

With sample program.

python

import numpy as np
import cv2
from PIL import Image, ImageDraw, ImageFont

def cv2_putText_5(img, text, org, fontFace, fontScale, color, mode=0):
# cv2.putText()Original argument "mode" not found in org Coordinate standard
#0 (default) = cv2.putText()Same as lower left 1 = upper left 2 = center

    #Get text description area
    fontPIL = ImageFont.truetype(font = fontFace, size = fontScale)
    dummy_draw = ImageDraw.Draw(Image.new("RGB", (0,0)))
    text_w, text_h = dummy_draw.textsize(text, font=fontPIL)
    text_b = int(0.1 * text_h) #Countermeasures for the amount that protrudes below due to a bug

    #Get the upper left coordinates of the text description area (start from the upper left of the original image)
    x, y = org
    offset_x = [0, 0, text_w//2]
    offset_y = [text_h, 0, (text_h+text_b)//2]
    x0 = x - offset_x[mode]
    y0 = y - offset_y[mode]
    img_h, img_w = img.shape[:2]

    #Do nothing outside the screen
    if not ((-text_w < x0 < img_w) and (-text_b-text_h < y0 < img_h)) :
        print ("out of bounds")
        return img

    #Upper left and lower right of the area where the original image is located in the text description area (the origin is the upper left of the original image)
    x1, y1 = max(x0, 0), max(y0, 0)
    x2, y2 = min(x0+text_w, img_w), min(y0+text_h+text_b, img_h)

    #Create a black image of the same size as the text description area, and paste the original image on all or part of it
    text_area = np.full((text_h+text_b,text_w,3), (0,0,0), dtype=np.uint8)
    text_area[y1-y0:y2-y0, x1-x0:x2-x0] = img[y1:y2, x1:x2]

    #Convert it to PIL, specify the font and draw the text (no color conversion)
    imgPIL = Image.fromarray(text_area)
    draw = ImageDraw.Draw(imgPIL)
    draw.text(xy = (0, 0), text = text, fill = color, font = fontPIL)

    #Convert PIL image back to OpenCV image (no color conversion)
    text_area = np.array(imgPIL, dtype = np.uint8)

    #Update the corresponding area of the original image to the one with characters drawn
    img[y1:y2, x1:x2] = text_area[y1-y0:y2-y0, x1-x0:x2-x0]

    return img

def main():
    img = np.full((200,400,3), (160,160,160), dtype=np.uint8)
    imgH, imgW = img.shape[:2]

    fontPIL = "Dflgs9.TTC" #DF Reiga Song
    size = 30
    text = "Japanese too\n possible"
    color = (255,0,0)

    positions = [(-imgW,-imgH),                 #This is outside the image and is not depicted
                 (0,0), (0,imgH//2), (0,imgH),
                 (imgW//2,0), (imgW//2,imgH//2), (imgW//2,imgH),
                 (imgW,0), (imgW,imgH//2), (imgW,imgH)]

    for pos in positions:
        img = cv2.circle(img, pos, 60, (0,0,255), 3)
        img = cv2_putText_5(img = img,
                            text = text,
                            org = pos,          #You specified the same coordinates as the center of the circle
                            fontFace = fontPIL,
                            fontScale = size,
                            color = color,
                            mode = 2)           #The coordinates you just specified are the center of the character description area.

    cv2.imshow("", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

I made it myself, but the central standard is compatible with cv2.circle (), and it seems to be useful.

At the end

We would like to thank kounoike for commenting in the previous article and Qiita for providing this opportunity.