[PYTHON] I tried to make Othello AI with tensorflow without understanding the theory of machine learning ~ Battle Edition ~

Series table of contents

-I tried to make Othello AI with tensorflow without understanding the theory of machine learning-Introduction- -I tried to make Othello AI with tensorflow without understanding the theory of machine learning-Implementation- ――I tried to make Othello AI with tensorflow without understanding the theory of machine learning-Iza Battle- -I tried to make Othello AI after trying to understand the theory of machine learning ~ Restart! ~ -I tried to make Othello AI after trying to understand the theory of machine learning-What is this Alpha Zero edition- -I tried to make a neutral network with Excel to understand the theory of machine learning ~ Image recognition mnist edition ~

Continuation of Last time ... In this field, as an outsider, I didn't study "theory of machine learning" at all. I would like to make an AI for Othello.

Preparation

The opponent is here made by a friend ・ PHP Reversi

First, open the opponent's site and player white (select the second move)

Start the prepared AI

python

$ python FightWithAI.py

------------- GAME START ---------------
***user turn ○***
  0  1  2  3  4  5  6  7
  8  9 10 11 12 13 14 15
 16 17 18 19 20 21 22 23
 24 25 26 ○ ● 29 30 31
 32 33 34 ● ○ 37 38 39
 40 41 42 43 44 45 46 47
 48 49 50 51 52 53 54 55
 56 57 58 59 60 61 62 63
Please enter the number
[44, 37, 26, 19]

>>>

I will enter the AI instructions.

Battle

First move (opponent) No. 19

Gote (AI) No. 18

python

***AI turn ●***
  0  1  2  3  4  5  6  7
  8  9 10 11 12 13 14 15
 16 17 18 ● 20 21 22 23
 24 25 26 ● ● 29 30 31
 32 33 34 ● ○ 37 38 39
 40 41 42 43 44 45 46 47
 48 49 50 51 52 53 54 55
 56 57 58 59 60 61 62 63
Please enter the number
[44, 37, 26, 19]

>>> 18

First move (opponent) No. 26

Gote (AI) No. 34

First move (opponent) No. 42

Gote (AI) No. 44

First move (opponent) No. 43

Gote (AI) No. 12

First move (opponent) No. 29

Gote (AI) No. 37

First move (opponent) No. 21

Gote (AI) No. 22

First move (opponent) No. 23

Gote (AI) No. 15

First move (opponent) No. 7

Gote (AI) No. 41

First move (opponent) No. 3

Gote (AI) No. 14

First move (opponent) No. 5

Gote (AI) No. 6

First move (opponent) No. 20

Gote (AI) No. 4

First move (opponent) No. 45

Gote (AI) No. 46

First move (opponent) No. 11

Gote (AI) No. 2

First move (opponent) No. 1

Gote (AI) No. 13

First move (opponent) No. 17

Gote (AI) No. 31

First move (opponent) No. 39

Gote (AI) No. 16

First move (opponent) No. 25

Gote (AI) No. 24

First move (opponent) No. 32

Gote (AI) No. 33

First move (opponent) No. 8

Gote (AI) No. 9

First move (opponent) No. 0

Gote (AI) No. 10

First move (opponent) No. 40

Gote (AI) pass

First move (opponent) No. 47

Gote (AI) No. 49

First move (opponent) No. 56

Gote (AI) No. 53

First move (opponent) No. 48

Gote (AI) No. 57

First move (opponent) No. 58

Gote (AI) No. 55

First move (opponent) No. 63

Gote (AI) No. 52

First move (opponent) No. 62

Gote (AI) No. 61

First move (opponent) No. 54

Gote (AI) No. 38

First move (opponent) 59th

Gote (AI) No. 30

First move (opponent) No. 60

Gote (AI) No. 51

First move (opponent) No. 50

result

Eh, it ’s a complete defeat orz Or rather, I spent a month making the weakest AI that lets me take all four corners.

Even at the end of the training, Q_MAX was 0.023, which was too small, so I thought it was strange. Normally, at the end of training, the log should look like this and Q_MAX should be close to 1.

python

player:1 | pos:32 | LOSS: 0.0014 | Q_MAX: 0.9423
player:2 | pos:15 | LOSS: 0.0020 | Q_MAX: 0.9335
…

Entertainment AI is ready

Revenge

Next time Revenge by changing from tensorflow to chainer.

Apparently optimizers.RMSpropGraves, which cannot be used with tensorflow, is working I've been learning well so far. (At the time of writing, it's not strong yet)

If you are familiar with this, please let us know. m(_ _)m