[PYTHON] [OpenCV] ArUco marker detection sample script

Introduction

OpenCV AR marker library ArUco Python3 sample script

Sample script

I put the source on github. OpenCV_SamplesForMyself

Reference link

-[Open CV] OpenCV drawing function

Preparation

environment

I used Windows10, Python 3.8.6. When using streaming, I used the Raspberry Pi 3 model B with a camera module. Raspberry Pi can be streamed with MJPEG-Streamer.

OpenCV installation

If you can use Python3, you should be able to use OpenCV by installing it with the following command.

$ pip install opencv-contrib-python

Creating an AR marker image

If you execute the following script, the AR marker will be output (ar.png) You can also output other markers by executing opcv_outputARmark02.py on github.

opcv_outputARmark01.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import cv2
aruco = cv2.aruco
dictionary = aruco.getPredefinedDictionary(aruco.DICT_4X4_50)

def arGenerator():
    fileName = "ar.png "
    # 0:ID number, 150x150 pixels
    generator = aruco.drawMarker(dictionary, 0, 150)
    cv2.imwrite(fileName, generator)
    img = cv2.imread(fileName)

arGenerator()

Script execution command

$ python opcv_outputARmark01.py

The output marker ar.png.

Sample script overview

This is an overview of the data stored on github.

--Column description --Detection target --Target to detect AR marker --Image --Use PNG format image --Video --Includes the following 3 formats - --Video in MP4 format --Streaming (Raspberry Pi + camera) --Camera device (Windows PC + camera) --Drawing --Type of information to be given to the detected AR marker -(ID, text, images, videos, etc.) --Output --Format to output the execution result to a file -(Image, video) --Dummy file --A dummy text file is stored instead of the mp4 file. - (dummy)test-mapping.mp4____.txt --Dummy file of test-mapping.mp4 - https://youtu.be/S-h031SBLaQ - (dummy)test-video.mp4____.txt --test-video.mp4 dummy file - https://youtu.be/qlqU_y5hu0k --Video data can be downloaded here - https://note.com/agw/n/na2f22f876228

• Image file --Image for AR marker detection - test-img01.png - - test-img02.png - - test-img03.png - --Image for mapping - test-mapping.png -

Script to detect markers from images

[IMG-01] Draw ID: opcv_outputARmark01.py

For all AR markers detected by aruco.drawDetectedMarkers (img, corners, ids, (0,255,0)), draw a rectangle with the ID and the four corners of the marker as vertices.

[IMG-02] Draw characters: opcvCapImg02_drawText.py

Draw characters with cv2.putText (...). In the sample, the coordinates of the four corners of the marker are drawn.

[IMG-03] Draw Line: opcvCapImg03_drawLine.py

Draw a straight line with cv2.line (...).

[IMG-04] Draw Rectangle: opcvCapImg04_drawRectangle.py

Draw a rectangle with cv2.rectangle (...).

[IMG-05] Draw a circle: opcvCapImg05_drawCircle.py

Draw a circle with cv2.circle (...).

[IMG-06] Draw Polygon: opcvCapImg06_drawPolylines_outImg.py

Draw a polygon with cv2.polylines (...).

[IMG-07] Draw an image

[IMG-08] Draw 3D axis

Script to detect markers from video

• Introducing the execution result using the video test-video.mp.

[VIDEO-01] Draw ID: opcvCapVideo01_drawID.py

Detects the marker and draws the ID. Unlike other video samples, the result is not output as a video. https://youtu.be/XVpjjGrzeAw

[VIDEO-02] Draw ID and output video file: opcvCapVideo02_drawID_outMP4.py

Similar to [VIDEO-01], it detects the marker and draws the ID.

https://youtu.be/XVpjjGrzeAw

[VIDEO-03] Draw a rectangle and output a video file: opcvCapVideo03_drawRectangle_outMP4.py

https://youtu.be/eyuDafmZQ28

[VIDEO-04] Draw characters and output video file: opcvCapVideo04_drawText_outMP4.py

https://youtu.be/zTJSGT5dQ_g

[VIDEO-05] Draw 3D axis and output video file: opcvCapVideo05_drawAxis_outMP4.py

https://youtu.be/nyaPeCBZ2hQ

[VIDEO-06] Draw an image and output a video file: opcvCapVideo06_drawImg_outMP4.py

https://youtu.be/1aR9-r7BFjQ

[VIDEO-07] Draw a video and output a video file: opcvCapVideo07_drawVideo_outMP4.py

Draw a video file (MP4) on the detected marker. It seems that the output video does not include the video drawn according to the marker (?), And the output video did not draw correctly.

https://youtu.be/jqJlxl5lzkI

When using a camera device or streaming

cv2.VideoCapture (...) is also compatible with camera devices and MJPEG-Streamer. Use cv2.VideoCapture (0) to specify the camera device and cv2.VideoCapture ("http: // {IP Address}: 8090 /? Action = stream ") to specify streaming.

You will be able to detect/draw markers for streaming as shown in the figure below.

I have confirmed that the camera device and streaming can be used in the video sample script of here.

Here's an example of using a camera device or streaming with a script in here. opcvCapVideo07_drawVideo_outMP4.py can be used by changing the values ​​of the variables targetVideo and outputVideo excerpted below.

opcvCapVideo07_drawVideo_outMP4.py

# ---Excerpt--- #

#targetVideo = 0 #Camera device
#targetVideo = "test-video.mp4" #Video
#targetVideo = "http://{IP Address}:8090/?action=stream" # MJPG-Streamer
targetVideo = "test-video.mp4"

#mappingVideo= 0 #Camera device
mappingVideo= "test-mapping.mp4" #Video
#mappingVideo= "http://{IP Address}:8090/?action=stream" # MJPG-Streamer
outputVideo= "editV07.mp4"

# ---Excerpt--- #

Example) Sample when a marker is detected from streaming and an image is drawn

In 2021, the first work is "infinity mirror AR". #protoout #opencv pic.twitter.com/F1kZA6JtyT

& mdash suo-takefumi (@zsipparu) January 1, 2021

Example) Sample when a marker is detected from streaming and a video (MP4) is drawn

& quot; Video version of the infinity mirror AR & quot;
Cows are wandering around. # 丑 # protoout #opencv #AR pic.twitter.com/FPn5ZWqvO1

& mdash; suo-takefumi (@zsipparu) January 1, 2021

Reference: Building a streaming server

I referred to here for the installation of MJPG-Streamer. https://qiita.com/suo-takefumi/items/2ae5527869dc13d038a9

Summary

I investigated how to use OpenCV, so I summarized the information. It seems that OpenGL can handle 3D objects as well, so I'll investigate that as well.