[PYTHON] The story of displaying images with OpenCV or PIL (only)

Introduction

The story of OpenCV and PIL. There is a lot of information on the web, not just Qiita, but many of the articles are fragmented and not suitable for comparison, so I tried to summarize it with that in mind. In the process, I came up with something that my predecessor didn't touch on (it seems), so I hope you'll read it.

OpenCV CV is an abbreviation for Computer Vision, which has various functions other than simply processing images. Well, this time it's just about displaying an image rather than processing it.

Read image cv2.imread ()

Use cv2.imread (* filename *, * flags *).

• filename * is the filename. If this is not appropriate, it will not result in an error and will return None.
• flags * will be described later.

Image loading

import cv2

filename = "hoge.png "
imgCV = cv2.imread(filename)  #flags omitted (default value = 1)

Display image cv2.imshow ()

Use cv2.imshow (* winname *, * mat *).

• winname * is the name of the window. It can be a null string, but it must be specified.
• mat * is a matrix, or matrix. In short, image data. You can't resize it by default, but you can resize it by saying "I'll make a resizable window with this name" in cv2.namedWindow () before doing cv2.imshow (). If cv2.imread () fails and None is returned, an error will occur here.

Image display in a non-resizeable window

cv2.namedWindow("image", cv2.WINDOW_AUTOSIZE)  #This sentence does not have to be
cv2.imshow("image", imgCV)

Image display in a resizable window

cv2.namedWindow("image", cv2.WINDOW_NORMAL)  # cv2.WINDOW_Since the value of NORMAL is 0, 0 may be specified.
cv2.imshow("image", imgCV)

Postscript: Full screen display

To display in full screen, first make the window size changeable, and then actually set it to full screen.

Functions that display or not display in full screen

def cv2_imshow_fullscreen(winname, img, flag_fullscreen):
    if flag_fullscreen:
        cv2.namedWindow(winname, cv2.WINDOW_NORMAL)
        cv2.setWindowProperty(winname, cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_FULLSCREEN)
    cv2.imshow(winname, img)

#Usage example
cv2_imshow_fullscreen("fullscreen", img , True) #Full screen view
cv2_imshow_fullscreen("window", img , False) #Window display

I tried to write a function for both, but if you do not want to display full screen, you can do cv2.imshow () normally, so it was better to make it a function dedicated to full screen display that does not branch with if ..

About OpenCV image data

The image data is composed of the type of numpy.ndarray. You can check the dimensions by shape and it is easy to check the contents. Note that ** colors are stored in BGR order **.

Image shape

Let's check what the image data looks like when the image is read by specifying flags.

flags	cv2.IMREAD_COLOR	cv2.IMREAD_GRAYSCALE	cv2.IMREAD_UNCHANGED
value	1 （デフォルトvalue）	0	-1
processing	Read as a color image	Read in grayscale	Read with the same specifications
Example 1 RGBA image <fontsize="-1">Theactualbackgroundistransparentratherthancheckered	shape=(200,182,3)	shape=(200,182)	shape=(200,182,4)
Example 2 Grayscale image	shape=(192,144,3)	shape=(192,144)	shape=(192,144)

In cv2.IMREAD_COLOR, the number of color channels is uniformly arranged in 3 channels regardless of whether the image is transparent or grayscale. In the case of cv2.IMREAD_GRAYSCALE, the number of channels is not 1, but it is a two-dimensional array with no specified number of channels (h, w). cv2.IMREAD_UNCHANGED is [OpenCV-Python tutorial "Handling images"](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_gui/py_image_display/py_image_display .html # id3) says "Read as an image including the alpha channel", but it does not have 4 channels uniformly, and it is correct to read from the original image without change. 4 channels for RGBA images, 3 channels for RGB images. Since it is one channel for a grayscale image, it returns (h, w) instead of (h, w, 1). Is it complicated? No, you said that from the beginning, cv2.IMREAD _ ** UNCHANGED **.

Get size

OpenCV image data whose array shape differs depending on the type of original image. You don't have to use an if statement to separate cases to get the height and width. The shape of the grayscale image obtained by (height, width) and the color image obtained by (height, width, number of channels) are the same as having the 0th height and the 1st width. That's why you should decide with 0 or 1.

python

#Either is acceptable

def getSize1(imgCV):
    h = imgCV.shape[0]
    w = imgCV.shape[1]
    return h, w

def getSize2(imgCV):
    h, w = imgCV.shape[:2]
    return h, w

For Google Colab

Cv2.imshow () cannot be used in Google Colab. It seems that the Jupyter session crashes. Instead, it offers an alternative to use Google Colab's own method called cv2_imshow (). No window name is required for cv2_imshow (), only image data is specified. This will display the image in the cell of Colab.

Google Colab

#1 cell 1 sentence or all can be put together
import cv2
from google.colab.patches import cv2_imshow

filename = "hoge.png "
imgCV = cv2.imread(filename)

cv2_imshow(imgCV)

For Jupyter Notebook

Cv2.imshow () isn't forbidden in Jupyter Notebook, but it can still crash.

In fact, in Jupyter Notebook, it can be displayed correctly by performing the correct processing. Cv2. After displaying with imshow (), you can wait for the key input and destroy the window. Even in this case, do not try to close the image window with the upper right batten. After all it crashes. Even if it can be displayed correctly, I hate to die if I make a mistake. The game world alone is enough for that.

Jupyter Notebook

import cv2

filename = "hoge.png "
imgCV = cv2.imread(filename)

#Do the following in one cell
cv2.imshow("image",imgCV)
cv2.waitKey(0)
cv2.destroyAllWindows()

What does correct processing mean?

I wrote that it is cv2.imshow () to display the image, but some people may have already stumbled at this stage. It works fine when run on the development environment IDLE that comes with Python when it is installed. But when I double-click the py file, it doesn't work as expected. Not even VS Code. This is because IDLE keeps the shell alive even after the program ends, but when you run python.exe, the image window closes the moment it ends. It seems to be. ** Destroy windows created with cv2 with cv2. ** This is justice in this world.

PIL(Pillow) There is an image processing library called PIL (Python Image Library), and the successor is Pillow. The difference between the strengths and weaknesses of OpenCV is coming again. Even if Pillow is installed, it is PIL that is imported when actually using it.

Load image Image.open ()

Use it as Image.open (filename). An error will occur if filename is incorrect. Strictly speaking, there is also an argument called mode, and the default value is "r", but I'm not sure what this means and what other values can be used.

Display image show ()

The image is displayed by showing () the read image data. Images are displayed by starting a different image viewer for each OS. It's a little inconvenient. Arguments include title and command, but both can be omitted. Needless to say, parentheses are required even if there are no arguments.

Source

from PIL import Image

filename = "hoge.png "
imgPIL = Image.open(filename)  #Image loading

imgPIL.show()  #Image display

About PIL image data

For example, if the image data is a png image, it is in the format of PIL.PngImagePlugin.PngImageFile, and it is not easy to check the contents. Instead, it has various attributes because it is aware that it is image data.

Get size

python

print (imgPIL.mode)
# RGBA  #There are also RGB L (gray scale).
# Image.open()The relationship with the mode of is unknown.

print (imgPIL.size)
# (182, 200)  #Tuple, width,In order of height

print (imgPIL.width)
# 182

print (imgPIL.height)
# 200

Displayed as a matplotlib graph

Images are often displayed as matplotlib graphs. Detailed usage of matplotlib.pyplot is not explained here.

When run on python, an interactive matplotlib graph appears. You can enlarge or change the display area.

Graphs are displayed as simple images on Google Colab and Jupyter Notebook. In Jupyter Notebook, it seems good to chant the magic of% matplotlib inline.

e? Can't you see the difference from displaying an image normally? So what about this image? 　← ここにいる This is a 6x8 image. I'm grateful that the graph of matplotlib enlarges such a small image nicely.

For PIL images

Since the PIL image cannot be graphed as it is, it is necessary to make it numpy.ndarray with numpy.asarray ().

Source

import numpy as np
from PIL import Image
from matplotlib import pyplot as plt
%matplotlib inline  #Inline display in Jupyter Notebook

filename = "hoge.png "
imgPIL = Image.open(filename)
arrPIL = np.asarray(imgPIL)
plt.imshow(arrPIL)
plt.show()

For OpenCV

Only here, due to various circumstances, the sample image is not a little skiman nurse.

This is the original image.

Numpy.ndarray as well as OpenCV image data. Then plt.imshow should be done as it is? When I try.

Yes no. You said that the OpenCV image is BGR. Since matplotlib.pyplot is usually RGB, it is necessary to convert the color when displaying an OpenCV image as a graph with matplotlib.

Use cv2.cvtColor to convert colors. Use cv2.cvtColor (src, code). src is the source. Original image data. code is a built-in constant for color conversion. BGR is RGB, vice versa, RGB is gray, RGB is RGBA, and so on. It is cv2.COLOR_BGR2RGB that converts BGR to RGB. With this effort, OpenCV images can also be displayed as matplotlib graphs.

BGR → RGB is nothing but the reverse order of RGB in the second array of shapes (height, width, BGR value). You can also take advantage of Last Learned Slice.

Source

import numpy as np
import cv2
from matplotlib import pyplot as plt

filename = "nurse.jpg "
imgCV = cv2.imread(filename)

# cv2.How to use cvtColor
imgCV_RGB = cv2.cvtColor(imgCV,cv2.COLOR_BGR2RGB)

#How to use slices
# imgCV_RGB = imgCV[:, :, ::-1]

plt.imshow(imgCV_RGB)
plt.show()

I didn't use transparent png images here because I will investigate in detail in the future. And it's not because I couldn't do it, don't get me wrong. As proof, an example in which transparency can be correctly expressed as a graph image is shown.

Next time preview

When the image and the image are combined, the transparent part of the front image makes the back image transparent. If you are a gamer in the 80's, you will manually perform mask processing that makes you want to say "Oh, sprite".

Finally, let's look again at the table that displays transparent images with different flags. ~~ I did my best to make it ~~ As a preparation.

The original image	cv2.IMREAD_COLOR	cv2.IMREAD_GRAYSCALE	cv2.IMREAD_UNCHANGED