Introduction

Annotation (teacher label creation) is the most important but troublesome part of machine learning. For image recognition, annotation tools such as VoTT are available, but it is still troublesome to click with the mouse. It would be fun if we could create labels automatically to some extent and fix only the strange parts.

Active Learning Function (VoTT)

In fact, there are functions in the world that can fulfill such demands, such as VoTT's Active Learning function. Introduction of Active Learning function of VoTT --Qiita

However, the trained models that can be used are fixed, and some tasks may not be available. I and others want to Annotate the Marufuku sign, but ** the existing model for object detection is refreshing. I mean, that's why I'm trying to learn the model myself. ** **

So, I will show you how to convert labels created using your favorite logic and model so that they can be read by annotation tools. For example, it is possible to read the following labels created with OpenCV with an annotation tool, modify only the necessary parts, and use them for deep learning.

OpenCV shape detection - PyImageSearch

Verification environment

Windows 10 Home (1903)
Python 3.6.8 --pillow == 7.1.2 (installed from pip)
VoTT 1.7.2

VoTT 2.1.0 was released as of May 2020 when I wrote the article, but it seemed difficult to load the annotations I made because it became too sophisticated and the structure of the project was complicated. Therefore, this time we will load it with the older version 1.7.2.

procedure

VoTT installation

Use VoTT 1.7.2. Download and install vott-win.exe. Release 1.7.2 · microsoft/VoTT

Data preparation

Create a folder in a suitable location and put the images (JPG or PNG) you want to annotate directly under it. Here, the full path of the folder is C: \ foo \ bar \ myproj. For VoTT 1.7.2, the name of the corresponding project file is C: \ foo \ bar \ myproj.json.

Creating automatic annotation results

`make_vott_project.py`


import sys
import json
import hashlib
import urllib
from pathlib import Path
from PIL import Image # pillow 7.1.2

imgdir = Path(sys.argv[1]).resolve()
projfile = imgdir.with_suffix(".json")

if projfile.exists():
    #Load an existing project
    f = open(projfile, "r+")
    data = json.load(f)
else:
    #Create a new project
    f = open(projfile, "w")
    data = {
        "frames": {},
        "framerate": "1",
        "inputTags": "mrfk", #Tag list (separated by commas)
        "suggestionType": "track",
        "scd": False,
        "visitedFrames": [],
        "tag_colors": ["#0cc7ff"] #Area color (optional)
    }

with f:
    for imgfile in imgdir.glob("*.*"):
        if imgfile.suffix.lower() in [".jpg ", ".png "]:
            if imgfile.name in data["frames"]:
                #Skip if there is an entry
                continue
            else:
                #If there is no entry, create a new one
                frame = []
                data["frames"][imgfile.name] = frame

            #Get the size of the image
            img = Image.open(imgfile)
            w, h = img.size
            img.close()

            #List the areas you have detected (provisional)
            points = [ #List the vertices in the order in which they are connected by edges
                {"x": 0.0, "y": 0.0},
                {"x": w,   "y": 0.0},
                {"x": w,   "y": h},
                {"x": 0.0, "y": h}
            ]
            box = {  #Circumscribed rectangle
                "x1": min(p["x"] for p in points),
                "y1": min(p["y"] for p in points),
                "x2": max(p["x"] for p in points),
                "y2": max(p["y"] for p in points)
            }
            region = box.copy()
            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "rect",
                "tags": ["mrfk"], #Tag list to be given to the area
            })
            frame.append(region)

    #Save project
    f.seek(0)
    json.dump(data, f)

If you execute the following from the command prompt, the tag mrfk will be added to the entire area of each image.

python make_vott_project.py C:\foo\bar\myproj

How to edit with VoTT

Launch VoTT 1.7.2, click the image icon and select the C: \ foo \ bar \ myproj folder. ** Please note that the project file name is automatically determined from the folder name. ** **

The following screen is OK with Continue as it is

A label is attached to the entire image. You can switch the image with the buttons of 2 left-pointing triangles and 2 right-pointing triangles.

If you want to modify the label, select Regions Manipulation in the upper left, then drag and drop the four corners of the area.

You can overwrite and save by selecting File → Save from the menu. You can read the updated json file and create training data appropriately.

Customize

Load annotations created externally such as OpenCV

#List the areas you have detected (provisional)

In the part of, enter the detection result in points. Specify the coordinates of each vertex of the area in the order in which they are connected by edges.

For example, if you want to use the code from the tutorial below OpenCV shape detection - PyImageSearch

frame = []
# loop over the contours
for c in cnts:
    (Abbreviation)
    points = [{"x": p[0], "y": p[1]} for p in c]
    (Abbreviation)
    frame.append(region)

You can make it like this.

Specify the area of the polygon

If you want to specify a complex area that is not a rectangle

            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "rect",
                "tags": ["mrfk"], #Tag list to be given to the area
            })

This part

            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "polygon", #Change here
                "tags": ["mrfk"], #Tag list to be given to the area
            })

Just change to. This way, when you move one vertex, the other vertices will not move. You can create a non-rectangular shape as shown below.

Make multiple tags

First, specify the name as a comma-separated string in place of `ʻinputTags`` below.

    data = {
        "frames": {},
        "framerate": "1",
        "inputTags": "mrfk,chst,wide", #Tag list (separated by commas)
        "suggestionType": "track",
        "scd": False,
        "visitedFrames": [],
        "tag_colors": ["#0cc7ff"] #Area color (optional)
    }

For tags in each area, specify a list of tags (a list of Python, not a comma-separated string).

            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "rect",
                "tags": ["mrfk", "chst"], #Tag list to be given to the area
            })

As you can see, it is reflected in the bottom left of the window and in the tooltip of the image.

If you want to further subdivide the existing object detection result class, you can increase only `ʻinputTags`` and edit the class (tag) of each area with VoTT.

Change the color of the tag

If you want to change the color of the area or the text color of the tag list at the bottom left, you can specify the color list in tag_colors below.

    data = {
        "frames": {},
        "framerate": "1",
        "inputTags": "mrfk,chst,wide", #Tag list (separated by commas)
        "suggestionType": "track",
        "scd": False,
        "visitedFrames": [],
        "tag_colors": ["#ff4040", "#ffff40", "#008000"] #Area color (optional)
    }

You will be able to put out in your favorite color as follows. (The image is after editing the annotation by myself)

[PYTHON] [Image recognition] How to read the result of automatic annotation with VoTT