Explanation of creating an application for displaying images and drawing with Python


Python is becoming more popular with the popularity of AI (Deep Learning). I have been making a GUI application that displays, draws, and saves images in Python, but I tried to organize the code and so on. I will write the mechanism to organize my understanding. I used a library called Qt for Python (PySide2). Originally developed in C ++, Qt allows you to develop cross-platform applications that run on various operating systems such as Windows, Mac and Linux from the same source code. There are Qt for Python or PyQt to use Qt from Python, but I used Qt for Python, which is not so bound by the license of the created application. When creating a GUI application, it is easy to understand if you understand object orientation. When drawing an image, many objects appear, and it is difficult for beginners to understand the roles and relationships of each object, but it will be easier to understand if something that moves even if it is not in a beautiful shape is created. (Experience story) So, if you have an image you want to make, don't give up and try various trials and errors. I hope this article will help you at that time.

The big picture of the app

The overall picture of the app is as shown below. The screen configuration is managed by the one that has a function called Layout that automatically arranges the placed parts (Widgets) according to the size of the window.


QVBoxLayout () aligns vertically and QHBoxLayout () aligns horizontally. Use QFormLayout () when you want to make a pair like a name and its value. The usage example is as follows.

self.main_layout = QVBoxLayout()
#Set the image display area
self.graphics_view = QGraphicsView()
#Nest the layout at the top of the screen into the main layout
self.upper_layout = QHBoxLayout()

Creating the main window

Create a class MainWindow (QMainWIndow) that inherits QMainWindow as the main window. Place parts (Widgets) with various functions in this, and describe the operation when you press it. Here, in the initialization, Layout could not be set unless it was declared as self.mainWidget = QWidget (self).

The code for launching the application looks like this:

class MainWindow(QMainWindow):

    def __init__(self):
    #Below, various processes are described

if __name__ == '__main__':
    app = QApplication(sys.argv)
    window = MainWindow()

Creating a menu bar

Declare it in the constructor (def init ()) to create a menu bar (selection at the top of the application) in the main window. To create a menu bar and create a menu called "File" in it, do the following.

class MainWindow(QMainWindow):

    def __init__(self):
        self.main_menu = self.menuBar()
        self.file_menu = self.main_menu.addMenu('File')

If you want to create an item called "File Open" in the "File" menu, do as follows.

# Set "Original Image Open" menu
self.org_img_open_button = QAction(self.style().standardIcon(getattr(QStyle, 'SP_FileDialogStart')), 'Open Orginal Image', self)

In the first line, it is set as QAction () that works when this item is selected, the icon is selected from the standard one, and the displayed name is'Open Orginal Image'. The second line is set so that this item can be selected with the Ctrl + O shortcut key. The third line is connected to the function that sets the behavior when this item is selected. This is explained in the next section. This item is registered in file_menu on the 4th line.

Mechanism for receiving actions on parts

In Qt, user operations and corresponding computer reactions are performed by methods called Signal and Slot. Signal is issued when the user performs an operation such as a menu button or dragging the mouse. Then, perform the corresponding processing in the Slot defined to receive each signal. Specifically, the operation when an item on the menu bar is selected is written as follows.

#menu bar->File-> 'Original Image Open'When is selected
#Send a triggered signal. The signal is open_org_img_Connected to the dialog function.
self.org_img_open_button = QAction(self.style().standardIcon(getattr(QStyle, 'SP_FileDialogStart')), 'Open Orginal Image', self)

#menu bar->File-> 'Original Image Open'A function that becomes a Slot that receives the signal emitted when is selected
def open_org_img_dialog(self):
    options = QFileDialog.Options()
    org_img_default_path = self.app_setting["SoftwareSetting"]["file_path"]["org_img_dir"]
    self.org_img_file_path, selected_filter = QFileDialog.getOpenFileName(self, 'Select original image', org_img_default_path, 'Image files(*.jpg *jpeg *.png)', options=options)
    org_img_dir_path, org_img_file = os.path.split(self.org_img_file_path)
    org_img_bare_name, org_img_ext = os.path.splitext(org_img_file)


When a color is selected with a mouse click from the color bar described later, the selected color is drawn as a rectangle in the drawing area for display to make it easier for the user to understand. At this time, it becomes necessary to act between the color bar and the object in the selected color drawing area. At that time, it is necessary to connect Signal and Slot through the parent main window. In that case, the class that manages the drawing items to be placed in the selected color drawing area defines its own Signal and emits the Signal with the color information of the clicked part. Specifically, the code looks like the one below.

# Class for graphics contents of tools on main window
class GraphicsSceneForTools(QGraphicsScene):
    # Define custom signal
    img_info = Signal(QColor)

    def mousePressEvent(self, event):
        # For check program action
        pos = event.scenePos()
        x = pos.x()
        y = pos.y()
        #If the cursor or pen is selected on the image editing toolbar, the color information of the clicked location(QColor)Issue a Signal with
        if self.mode == 'cursor' or self.mode == 'pen':
            self.pix_rgb = self.img_content.pixelColor(x, y)

#Define the Slot corresponding to the Signal of the color information selected on the main window side.
class MainWindow(QMainWindow):
    #Drawing item management object for color bars
    self.color_bar_scene = GraphicsSceneForTools()
    #Connect to the function that becomes the Slot that receives the Signal that sent the color information

Image display mechanism

Several widgets are linked for image display. The relationship is as shown in the figure below. Prepare QGraphicsView, which is a drawing area object, in MainWindow, place QGraphicsScene that holds and manage drawing objects in it, and add drawings and images such as lines and circles to QGraphicsScene. To go.


The QGraphicsScene set in the main drawing area displays the pixel information of the displayed image in the status bar when the cursor tool is selected, and the layer above the image when the pen or eraser tool is selected. I will try to draw. In order to add such a function set by yourself, create a Graphics Scene that inherits QGraphic Scene as follows. By setting the parent drawing area QGraphicsView and its parent MainWindow in the initialization init function, the information obtained from each item of this GraphicsScene can be passed to the drawing area or window.

To be honest, at first I'm not sure about QGraphicsView and QGraphicsScene, but I thought it was complicated and troublesome to access and control the content! It seems that this is because the design is designed to meet the complicated demand of drawing within the visible range (drawable range) from different viewpoints even if the target content to be drawn does not change. For example, when the content to be drawn is larger than the drawing area, it may be displayed while changing the viewpoint with the scroll bar, or the 3D object may be displayed while changing the viewpoint.

class GraphicsSceneForMainView(QGraphicsScene):

    def __init__(self, parent=None, window=None, mode='cursor'):
        QGraphicsScene.__init__(self, parent)
        # Set parent view area
        self.parent = parent
        # Set grand parent window
        self.window = window
        # Set action mode
        self.mode = mode

        # mouse move pixels
        self.points = []

        # added line items
        self.line_items = []
        self.lines = []

        # added line's pen attribute
        self.pens = []
    def set_mode(self, mode):
        self.mode = mode

    def set_img_contents(self, img_contents):
        # image data of Graphics Scene's contents
        self.img_contents = img_contents

    def clear_contents(self):
        self.img_contents = None

    def mousePressEvent(self, event):
        # For check program action
        pos = event.scenePos()
        x = pos.x()
        y = pos.y()

        if self.mode == 'cursor':
            # Get items on cursor
            message = '(x, y)=({x}, {y}) '.format(x=int(x), y=int(y))

            for img in self.img_contents:
                # Get pixel value
                pix_val = img.pixel(x, y)
                pix_rgb = QColor(pix_val).getRgb()
                message += '(R, G, B) = {RGB} '.format(RGB=pix_rgb[:3])

            # show scene status on parent's widgets status bar

Link to QGraphicsView documentation Link to QGraphicsScene documentation

Image object

To place an image in QGraphicsScene, format it as QPixmap and use QGraphicsScene.addItem (QPixmap). However, in the QPixmap format, the information of each pixel cannot be acquired or rewritten, so keep it in the QImage format and convert it to QPixmap for drawing. To create a QImage from an image file, turn it into a QPixmap, and add it to the QGraphicsScene, the code looks like this:

#self refers to MainWindow
self.scene = GraphicsSceneForMainView(self.graphics_view, self)
self.org_qimg = QImage(self.org_img_file_path)
self.org_pixmap = QPixmap.fromImage(self.org_qimg)

To create a QImage of 8 bits (256 gradations) RGBA (A is transparency) in the sky, use the following code.

self.layer_qimg = QImage(self.org_img_width, self.org_img_height, QImage.Format_RGBA8888)

Link to QImage documentation Link to QPixmap documentation

Color bar settings

With reference to this article, the color bar for selecting the pen color is heat map-like (smooth from low-temperature blue to high-temperature red). I made it with (change). The object that sets the color bar is partly described in the explanation of "Mechanism for receiving actions on parts", but I created a class called GraphicsSceneForTools that inherited QGraphicsScene and used it. By doing so, by clicking the mouse on it, Signal will be issued according to the position pressed from that object, and in the MainWindow (to be exact, MainWindow-> QGraphicsView-> GraphicsSceneForTools) of the parent object where the object is placed. By preparing a Slot function to be received by Signal, the selected color display area is filled with the color selected by the user from the color bar and displayed in an easy-to-understand manner. In the newly prepared class GraphicsSceneForTools (QGraphicsScene), a Signal with QColor (color information) is prepared as img_info = Signal (QColor), and it is set when the mouse is clicked in def mousePressEvent (self, event). I am trying to output a Signal signal with the color (self.pix_rgb) of the clicked coordinate position of the drawing item (color bar in this case) as self.img_info.emit (self.pix_rgb). On the MainWindow side, set_selected_color () is prepared as the Slot function on the receiving side when the GraphicsSceneForTools object issues the corresponding Signal as self.color_bar_scene.img_info.connect (self.set_selected_color). Specifically, the code is as follows.

class MainWindow(QMainWindow):
    # Set color bar
    self.color_bar_width = 64
    self.color_bar_height = 256
    self.color_bar_view = QGraphicsView()
    self.color_bar_view.setFixedSize(self.color_bar_width+3, self.color_bar_height+3)
    self.color_bar_scene = GraphicsSceneForTools()

    #Set the color bar.
    #The color change data that is the basis of the color bar is self.colormap_It is in data.
    #Please refer to the source code or reference article for how to create it.
    self.color_bar_img = QImage(self.color_bar_width, self.color_bar_height, QImage.Format_RGB888)

    for i in range(self.color_bar_height):
        # Set drawing pen for colormap
        ii = round(i * (1000/256))
        color = QColor(self.colormap_data[ii][0], self.colormap_data[ii][1], self.colormap_data[ii][2])
        pen = QPen(color, 1, Qt.SolidLine, \
                Qt.SquareCap, Qt.RoundJoin)
        self.color_bar_scene.addLine(0, self.color_bar_height - i-1, self.color_bar_width, self.color_bar_height - i-1, pen=pen)
        for j in range(self.color_bar_width):
           self.color_bar_img.setPixelColor(j, self.color_bar_height-i-1, color)



    # Connect signal to slot of color_bar_scene

    # Slot of color bar clicked for selection color
    def set_selected_color(self, color):
        # Delete existng image item
        self.draw_color = color
        brush = QBrush(self.draw_color)
        self.select_color_rect = self.select_color_scene.addRect(QRect(0, 0, self.select_color_view_size, self.select_color_view_size), \

# Class for graphics contents of tools on main window
class GraphicsSceneForTools(QGraphicsScene):
    # Define custom signal
    img_info = Signal(QColor)

    def __init__(self, parent=None, window=None):
        QGraphicsScene.__init__(self, parent)
        # Set parent view area
        self.parent = parent
        # Set grand parent window
        self.window = window
        self.mode = 'cursor'

    def set_mode(self, mode):
        self.mode = mode

    def set_img_content(self, img_content):
        # image data of Graphics Scene's contents
        self.img_content = img_content
    def mousePressEvent(self, event):
        # For check program action
        pos = event.scenePos()
        x = pos.x()
        y = pos.y()

        if self.mode == 'cursor' or self.mode == 'pen':
            self.pix_rgb = self.img_content.pixelColor(x, y)

Reflect the drawn contents and save as a file

With the above contents, you will be able to select a pen or eraser tool to draw. The original image is displayed, and the result of my drawing is drawn on another layer above it. Furthermore, in order to save the contents drawn by yourself, it is necessary to export the drawn contents by dragging the mouse as an image. The contents drawn by the user's mouse drag are saved as a collection of lines that are loci. Lines have start and end points and pen attributes (color, size). Therefore, the coordinates that pass on the image are calculated based on the start point and end point in order from the group of lines, and are reflected in the image for export.

#Class that manages items to be placed in the drawing area
class GraphicsSceneForMainView(QGraphicsScene):
    #When a pen or eraser is selected and drawn with mouse drag
    def mouseMoveEvent(self, event):
        # For check program action
        pos = event.scenePos()
        x = pos.x()
        y = pos.y()
        if self.mode == 'pen' or self.mode == 'eraser':
            if x >= 0 and x < self.width() and y >= 0 and y < self.height():
                if len(self.points) != 0:
                    draw_color = self.window.draw_color
                    # Set transparenc value
                    draw_size = self.window.draw_tool_size
                    pen = QPen(draw_color, draw_size, Qt.SolidLine, Qt.RoundCap, Qt.RoundJoin)
                    self.lines_items.append(self.addLine(QLineF(self.points[-1].x(), self.points[-1].y(), x, y), pen=pen))
                    #The position of the drawn line and the attributes of the pen at that time for later saving as an image(Color, size)Save


# Main Window components
class MainWindow(QMainWindow):
    #Processing to reflect the user's drawing on the image
    #The content drawn by dragging the mouse is a collection of line information with a start point and an end point.
    def make_layer_image(self):
        for i, line in enumerate(self.scene.lines):
            pen = self.scene.pens[i]

            pen_size = int(pen.width())
            pen_color = pen.color()

            # start pixel of line
            x1 = int(line.x1())
            y1 = int(line.y1())

            # end pixel of line
            x2 = int(line.x2())
            y2 = int(line.y2())

            dx = int(line.dx())
            dy = int(line.dy())

            # When only 1pixl line
            if dx <= 1 and dy <= 1:
                draw_pix_x1_s = max(x1 - int(pen_size/2), 0)
                draw_pix_x1_e = min(x1 + int(pen_size/2), self.org_img_width-1)
                draw_pix_y1_s = max(y1 - int(pen_size/2), 0)
                draw_pix_y1_e = min(y1 + int(pen_size/2), self.org_img_height-1)

                # for Pen's size
                for y in range(draw_pix_y1_s, draw_pix_y1_e):
                    for x in range(draw_pix_x1_s, draw_pix_x1_e):
                        self.layer_qimg.setPixelColor(x, y, pen_color)

                draw_pix_x2_s = max(x2 - int(pen_size/2), 0)
                draw_pix_x2_e = min(x2 + int(pen_size/2), self.org_img_width-1)
                draw_pix_y2_s = max(y2 - int(pen_size/2), 0)
                draw_pix_y2_e = min(y2 + int(pen_size/2), self.org_img_height-1)

                # for Pen's size
                for y in range(draw_pix_y2_s, draw_pix_y2_e):
                    for x in range(draw_pix_x2_s, draw_pix_x2_e):
                        self.layer_qimg.setPixelColor(x, y, pen_color)

                # For avoid devide by 0
                if dx == 0:
                    for y in range(y1, y2+1):
                        draw_pix_y_s = y - int(pen_size/2)
                        draw_pix_y_e = y + int(pen_size/2)
                        # for Pen's size
                        for yy in range(draw_pix_y_s, draw_pix_y_e):
                            self.layer_qimg.setPixelColor(x1, yy, pen_color)

                    grad = dy/dx

                    # Choose coordinates with small slope not to skip pixels
                    if grad >= 1.0:
                        for x in range(dx):
                            y = y1 + int(grad * x + 0.5)
                            draw_pix_x_s = max(x1 + x - int(pen_size/2), 0)
                            draw_pix_x_e = min(x1 + x + int(pen_size/2), self.org_img_width-1)
                            draw_pix_y_s = max(y - int(pen_size/2), 0)
                            draw_pix_y_e = min(y + int(pen_size/2), self.org_img_height-1)
                            # for Pen's size
                            for yy in range(draw_pix_y_s, draw_pix_y_e+1):
                                for xx in range(draw_pix_x_s, draw_pix_x_e+1):
                                    self.layer_qimg.setPixelColor(xx, yy, pen_color)

                        for y in range(dy):
                            x = x1 + int(1/grad * y + 0.5)
                            draw_pix_y_s = max(y1 + y - int(pen_size/2), 0)
                            draw_pix_y_e = min(y1 + y + int(pen_size/2), self.org_img_height-1)
                            draw_pix_x_s = max(x - int(pen_size/2), 0)
                            draw_pix_x_e = min(x + int(pen_size/2), self.org_img_width-1)
                            # for Pen's size
                            for yy in range(draw_pix_y_s, draw_pix_y_e+1):
                                for xx in range(draw_pix_x_s, draw_pix_x_e+1):
                                    self.layer_qimg.setPixelColor(xx, yy, pen_color)

Add'Save Layer Image' to the File menu and when you select it, it will save the image drawn by the user. Specifically, the code is as follows, execute the process to create a QImage image that reflects the drawing with make_layer_image () explained above, open the file dialog for saving, and save with the entered image file name. To do.

# Main Window components
class MainWindow(QMainWindow):
    def __init__(self):
        # Set "Save layer image" menu
        self.layer_img_save_button = QAction(self.style().standardIcon(getattr(QStyle, 'SP_FileDialogEnd')), 'Save Layer Image', self)

    # Slot function of save layer image button clicked
    def save_layer_image(self):


        layer_img_default_path = self.app_setting["SoftwareSetting"]["file_path"]["layer_img_dir"]
        options = QFileDialog.Options()
        file_name, selected_filete = QFileDialog.getSaveFileName(self, 'Save layer image', layer_img_default_path, \
            'image files(*.png, *jpg)', options=options)
        ret = QMessageBox(self, 'Success', 'layer image is saved successfully', QMessageBox.Ok)

Widget (part) placement in GUI

Qt also comes with a tool called Qt Designer that arranges parts such as buttons on the screen of the application you want to create on the GUI screen. Before you get used to it, it is easy to imagine what kind of parts (Widgets) you have, so it may be easier to understand if you try to make an appearance using this.

Source code

The source code of the created application will be posted in the following location. App source code page

Recommended Posts

Explanation of creating an application for displaying images and drawing with Python
Associate Python Enum with a function and make it Callable
Explanation of creating an application for displaying images and drawing with Python
Procedure for creating an application with Django with Pycharm ~ Preparation ~
Creating an egg with python
[Personal development] Story of creating an application for artists with SPA with Django REST Framework and Vue-Cli [DRF + Vue.js]
[For beginners] Summary of standard input in Python (with explanation)
Turn an array of strings with a for statement (Python3)
Coexistence of Python2 and 3 with CircleCI (1.0)
Easy-to-understand explanation of Python Web application (Django) even for beginners (5) [Introduction to DB operation with Django shell]
Amplify images for machine learning with python
Note when creating an environment with python
Capturing images with Pupil, python and OpenCV
[Python] Read images with OpenCV (for beginners)
Drawing with Matrix-Reinventor of Python Image Processing-
Importing and exporting GeoTiff images with Python
Get images of OpenStreetMap and Geographical Survey Institute maps with Python + py-staticmaps
Easy-to-understand explanation of Python Web application (Django) even for beginners (2) [Project creation]
Get images of OpenStreetMap and Geographical Survey Institute maps with Python + staticmap
I measured the speed of list comprehension, for and while with python2.7.
Easy-to-understand explanation of Python Web application (Django) even for beginners (1) [Environment construction]
Causal reasoning and causal search with Python (for beginners)
Building an Anaconda environment for Python with pyenv
Procedure for creating a LineBot made with Python
python: Tips for displaying an array (list) with an index (how to find out what number an element of an array is)
[Python] Collect images with Icrawler for machine learning [1000 images]
Implementation of TRIE tree with Python and LOUDS
[Python for Hikari] Chapter 09-02 Classes (Creating and instantiating classes)
Wavelet transform of images with PyWavelets and OpenCV
Continuation of multi-platform development with Electron and Python
Explanation of edit distance and implementation in Python
Example of reading and writing CSV with Python
Let's make an app that can search similar images with Python and Flask Part1
Build API server for checking the operation of front implementation with python3 and Flask
Extract images and tables from pdf with python to reduce the burden of reporting
Volume of creating and publishing django-malice, a django application for causing HTTP 40X errors
Easy-to-understand explanation of Python web application (Django) even for beginners (6) [MTV design pattern completion]
Flow of creating your own package with setup.py with python
Get media timeline images and videos with Python + Tweepy
Make your python CUI application an app for mac
Display embedded images of mp3 and flac with mutagen
List of Python libraries for data scientists and data engineers
Python netCDF4 read speed and nesting of for statements
Python --Explanation and usage summary of the top 24 packages
Try projective transformation of images using OpenCV with Python
Easy partial download of mp4 with python and youtube-dl!
Summary of tools for operating Windows GUI with Python
[# 2] Make Minecraft with Python. ~ Model drawing and player implementation ~
Parse and visualize JSON (Web application ⑤ with Python + Flask)
Building an environment for natural language processing with Python
Create a batch of images and inflate with ImageDataGenerator
Creating ρ method for elliptic curve cryptography with python
Creating BINGO "Web Tools" with Python (Table of Contents)
Python: Get a list of methods for an object
About creating and modifying custom themes for Python IDLE
Reading, displaying and speeding up gifs with python [OpenCV]
Get an abstract understanding of Python modules and packages
Explanation of package tools and commands for Linux OS
Comparison of CoffeeScript with JavaScript, Python and Ruby grammar
Version control of Node, Ruby and Python with anyenv
I tried "morphology conversion" of images with Python + OpenCV
Drawing with Python Tinker