[PYTHON] Updated Rubik's Cube Robot software 4. Status recognition

What is this article?

I am currently developing a robot that solves a 2x2x2 Rubik's cube. This is a collection of commentary articles on the robot program. I used to write an article collection represented by the article here, but since this time the software has been significantly updated so I will introduce a new program. think.

The corresponding code is available at here.

Related articles

"Let's make a robot that solves the Rubik's cube!"

1. Overview
2. Algorithm
3. Software
4. Hardware

Updated software for Rubik's Cube Robot

1. Basic function
2. Pre-calculation
3. Solution Search
4. State recognition (this article)
5. Machine operation (Python)
6. Machine operation (Arduino)
7. Main processing

This time, we will introduce `` `detector.py``` as a state recognition edition.

Import modules to use

Import the module you want to use.

import cv2
from time import sleep

from basic_functions import *
from controller import move_actuator

Here, `controller``` is the program that will be explained next time, but the `move_actuator``` function in this is a function that sends a command to move the actuator (here, the motor) as its name suggests.

Create 6 sides of information from 4 sides of the puzzle

With the 2x2x2 Rubik's Cube, you can guess the color of the entire surface just by looking at the colors on the four sides. To take advantage of this property, we need a function that infers the unfilled color from the filled color information. I will introduce the function.

Please let me apologize first. This function was originally created with momentum in the era of manually inputting colors (a few months ago), and I don't really know what I'm doing now! And now we can guess the colors of the remaining two sides by looking at the colors of a particular four sides, so it doesn't have to be such a versatile function. In other words, the function introduced here is ** a function that is a little redundant and I don't know what it is doing, but something is working **. sorry. Be careful of yourself, "at least add a comment."

As an aside, I am writing this article for myself in the future. I wrote a comment now and left a commentary so that I wouldn't be at a loss to read this code in the future.

'''Fill the place where the color is fixed where it is not filled'''
''' Fill boxes if the color can be decided '''
def fill(colors):
    for i in range(6):
        for j in range(8):
            if (1 < i < 4 or 1 < j < 4) and colors[i][j] == '':
                done = False
                for k in range(8):
                    if [i, j] in parts_place[k]:
                        for strt in range(3):
                            if parts_place[k][strt] == [i, j]:
                                idx = [colors[parts_place[k][l % 3][0]][parts_place[k][l % 3][1]] for l in range(strt + 1, strt + 3)]
                                for strt2 in range(3):
                                    idx1 = strt2
                                    idx2 = (strt2 + 1) % 3
                                    idx3 = (strt2 + 2) % 3
                                    for l in range(8):
                                        if parts_color[l][idx1] == idx[0] and parts_color[l][idx2] == idx[1]:
                                            colors[i][j] = parts_color[l][idx3]
                                            done = True
                                            break
                                    if done:
                                        break
                                break
                    if done:
                        break
    return colors

I will explain by guessing.

The `` `forstatement that turns the first twoi, j``` simply looks at the following array of colors, one at a time.

colors = [
    ['', '', 'w', 'o', '', '', '', ''],
    ['', '', 'w', 'g', '', '', '', ''],
    ['b', 'o', 'g', 'y', 'r', 'w', 'b', 'r'],
    ['o', 'o', 'g', 'g', 'w', 'r', 'b', 'b'],
    ['', '', 'y', 'r', '', '', '', ''],
    ['', '', 'y', 'y', '', '', '', '']
    ]

Where `` `w, y, g, b, o, r``` represent white, yellow, green, blue, orange and red, respectively. It should be noted that the way of holding this array is wasteful because it inherits the tradition of programs created in the past in a bad direction. I want to fix it someday.

Subsequent ```if (1 <i <4 or 1 <j <4) and colors [i] [j] =='': `` `where the color information is originally included or not I will divide it.

for k in range(8):Turn all possible parts candidates in, if[i, j] in parts_place[k]:If the two colors match with, for strt in range(3):Check all three orientations of the parts with.

 I think that the processing is done in this way.

# State recognition using a camera
 Use the camera to recognize the state of the puzzle. This time, we use a library called OpenCV to recognize colors.

```python
'''Get the state of the puzzle'''
''' Get colors of stickers '''
def detector():
    colors = [['' for _ in range(8)] for _ in range(6)]
    #Grab the puzzle
    for i in range(2):
        move_actuator(i, 0, 1000)
    for i in range(2):
        move_actuator(i, 1, 2000)
    sleep(0.3)
    rpm = 200
    capture = cv2.VideoCapture(0)
    #Each color(HSV)Range of
    #color: g, b, r, o, y, w
    color_low = [[40, 50, 50],   [90, 50, 70],    [160, 50, 50],   [170, 50, 50],    [20, 50, 30],   [0, 0, 50]]
    color_hgh = [[90, 255, 255], [140, 255, 200], [170, 255, 255], [10, 255, 255], [40, 255, 255], [179, 50, 255]]
    #Position of each part on the colors array
    surfacenum = [[[4, 2], [4, 3], [5, 2], [5, 3]], [[2, 2], [2, 3], [3, 2], [3, 3]], [[0, 2], [0, 3], [1, 2], [1, 3]], [[3, 7], [3, 6], [2, 7], [2, 6]]]
    #Other constants
    d = 10
    size_x = 130
    size_y = 100
    center = [size_x // 2, size_y // 2]
    dx = [-1, -1, 1, 1]
    dy = [-1, 1, -1, 1]
    #Load the four sides of the puzzle
    for idx in range(4):
        #Since there are times when reading does not go well, let the dummy read 5 times
        for _ in range(5):
            ret, frame = capture.read()
        tmp_colors = [['' for _ in range(8)] for _ in range(6)]
        loopflag = [1 for _ in range(4)]
        #While until all the colors of the four stickers on one side are loaded
        while sum(loopflag):
            ret, show_frame = capture.read()
            show_frame = cv2.resize(show_frame, (size_x, size_y))
            hsv = cv2.cvtColor(show_frame,cv2.COLOR_BGR2HSV)
            #Examine the four stickers in order
            for i in range(4):
                y = center[0] + dy[i] * d
                x = center[1] + dx[i] * d
                val = hsv[x, y]
                #Find out which of the 6 colors
                for j in range(6):
                    flag = True
                    for k in range(3):
                        if not ((color_low[j][k] < color_hgh[j][k] and color_low[j][k] <= val[k] <= color_hgh[j][k]) or (color_low[j][k] > color_hgh[j][k] and (color_low[j][k] <= val[k] or val[k] <= color_hgh[j][k]))):
                            flag = False
                    if flag:
                        tmp_colors[surfacenum[idx][i][0]][surfacenum[idx][i][1]] = j2color[j]
                        loopflag[i] = 0
                        break
        #Store values in the colors array
        for i in range(4):
            colors[surfacenum[idx][i][0]][surfacenum[idx][i][1]] = tmp_colors[surfacenum[idx][i][0]][surfacenum[idx][i][1]]
        #Turn the motor to see the next side
        move_actuator(0, 0, -90, rpm)
        move_actuator(1, 0, 90, rpm)
        sleep(0.2)
    capture.release()
    colors = fill(colors)
    return colors

The problem here is that the color range is definite. This can lead to color recognition problems due to disturbances. Actually, it didn't work well at the venue of Maker Faire Tokyo 2020 due to the light environment (I made a slight correction to the value and survived).

Summary

This time, I introduced the part that recognizes the state with the camera and restores the unseen surface. Next time, I will actually move the robot.