Einführung

―― Bisher haben wir mit CNN und RNN sogenanntes „überwachtes Lernen“ durchgeführt. ――Dieses Mal habe ich versucht, einfach "verbessertes Lernen" wie das berühmte AlphaGo zu machen. ――Aber es ist nur eine kleine Verbesserung für diejenigen, die es bereits geschafft haben. ―― ○ ✕ Das Spiel scheint offiziell Sanmoku zu heißen. (Ich wusste es nicht) ――Die Details stimmen mit dem Inhalt der Implementierung überein, daher werde ich sie nicht zweimal schreiben. ――Da es nicht kompliziert ist, ist es nicht schlank oder konstant.

Referenz

8. Machen wir AI für ○ × Spiel mit TensorFlow

Umgebung

#	OS/Software/Bibliothek	Ausführung
1	Mac OS X	EI Capitan
2	Python	2.7er Serie
3	TensorFlow	1.2 System

Einzelheiten

http://qiita.com/neriai/items/c0114af9c2eae627b6ce

Trainingsdaten

Ich benutze das so wie es ist. https://github.com/sfujiwara/tictactoe-tensorflow/tree/master/data

Quellcode

`ticktacktoo.py`


#!/usr/local/bin/python
# -*- coding: utf-8 -*-

import os
import shutil
import numpy as np
import tensorflow as tf

def inference(squares_placeholder):

    #Erstellen Sie die erste ausgeblendete Ebene
    with tf.name_scope('hidden1') as scope:
        hidden1 = tf.layers.dense(squares_placeholder, 32, activation=tf.nn.relu)

    #Erstelle versteckte Ebene 2. Ebene
    with tf.name_scope('hidden2') as scope:
        hidden2 = tf.layers.dense(hidden1, 32, activation=tf.nn.relu)

    #Erstellen Sie eine Ebene mit hoher Dichte
    with tf.name_scope('logits') as scope:
        logits = tf.layers.dense(hidden2, 3)

    #Normalisierung durch Softmax-Funktion
    with tf.name_scope('softmax') as scope:
        logits = tf.nn.softmax(logits)

    return logits

#Error(loss)Trainieren Sie ein Lernmodell, das mit Hilfe der Fehler-Backpropagation basierend auf entwickelt wurde
def loss(labels_placeholder, logits):

    cross_entropy = tf.losses.softmax_cross_entropy(
        onehot_labels=labels_placeholder,
        logits=logits,
        label_smoothing=1e-5
    )

    #Geben Sie an, dass in TensorBoard angezeigt werden soll
    tf.summary.scalar("cross_entropy", cross_entropy)

    return cross_entropy

#Error(loss)Trainieren Sie ein Lernmodell, das mit Hilfe der Fehler-Backpropagation basierend auf entwickelt wurde
def training(learning_rate, loss):

    #Wie diese Funktion macht das alles
    train_step = tf.train.AdamOptimizer(learning_rate).minimize(loss)

    return train_step

#Berechnen Sie die korrekte Antwortrate des Vorhersageergebnisses, das das Lernmodell bei Inferenz angibt
def accuracy(logits, labels):

    #Vergleichen Sie, ob das Vorhersageetikett und das richtige Antwortetikett gleich sind. Gibt True zurück, wenn sie identisch sind
    correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(labels, 1))

    #Boolescher Wert korrekt_Berechnen Sie die richtige Antwortrate, indem Sie die Vorhersage in float ändern
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

    #Auf TensorBoard anzeigen lassen
    tf.summary.scalar("accuracy", accuracy)

    return accuracy

if __name__ == '__main__':

    np.random.seed(1)

    mat = np.loadtxt('/workspace/tictactoe/data.csv', skiprows=1, delimiter=",")

    ind_train = np.random.choice(5890, 4000, replace=False)
    ind_test = np.array([i for i in range(5890) if i not in ind_train])

    train_square = mat[ind_train, :-1]
    test_square = mat[ind_test, :-1]

    all_label = np.zeros([len(mat), 3])

    for i, j in enumerate(mat[:, -1]):
        if j == 1:
            # x win
            all_label[i][0] = 1.
        elif j == -1:
            # o win
            all_label[i][1] = 1.
        else:
            # draw
            all_label[i][2] = 1.

    train_label = all_label[ind_train]
    test_label = all_label[ind_test]

    with tf.Graph().as_default() as graph:

        tf.set_random_seed(0)

        #Tensor zum Einfügen von Bildern(28*28*3(IMAGE_PIXELS)Beliebig viele dimensionale Bilder(None)Ich habe eine Minute)
        squares_placeholder = tf.placeholder(tf.float32, [None, 9])

        #Tensor, um ein Etikett zu setzen(3(NUM_CLASSES)Beliebig viele dimensionale Beschriftungen(None)Geben Sie die Minuten ein)
        labels_placeholder = tf.placeholder(tf.float32, [None, 3])

        #Generieren Sie ein Modell
        logits = inference(squares_placeholder)

        # loss()Den Verlust berechnen
        loss = loss(labels_placeholder, logits)

        # training()Trainieren und Anpassen der Parameter des Lernmodells
        train_step = training(0.01, loss)

        #Berechnung der Genauigkeit
        accuracy = accuracy(logits, labels_placeholder)

        #Bereit zum Speichern
        saver = tf.train.Saver()

        #Sitzung erstellen(TensorFlow-Berechnungen müssen in einer absoluten Sitzung durchgeführt werden)
        sess = tf.Session()

        #Variable Initialisierung(Nach dem Starten der Sitzung initialisieren)
        sess.run(tf.global_variables_initializer())

        #TensorBoard-Anzeigeeinstellungen(Tensor Board deklarativ?)
        summary_step = tf.summary.merge_all()

        # train_Geben Sie den Pfad für die Ausgabe des TensorBoard-Protokolls mit dir an
        summary_writer = tf.summary.FileWriter('/workspace/tictactoe/data', sess.graph)

        for step in range(10000):
            ind = np.random.choice(len(train_label), 1000)

            sess.run(
                train_step,
                feed_dict={squares_placeholder: train_square[ind], labels_placeholder: train_label[ind]}
            )

            if step % 100 == 0:
                train_loss = sess.run(
                    loss,
                    feed_dict={squares_placeholder: train_square, labels_placeholder: train_label}
                )

                train_accuracy, labels_pred = sess.run(
                    [accuracy, logits],
                    feed_dict={squares_placeholder: train_square, labels_placeholder: train_label}
                )

                test_accuracy = sess.run(
                    accuracy,
                    feed_dict={squares_placeholder: test_square, labels_placeholder: test_label}
                )

                summary = sess.run(
                    summary_step,
                    feed_dict={squares_placeholder: train_square, labels_placeholder: train_label}
                )

                summary_writer.add_summary(summary, step)

                print "Iteration: {0} Loss: {1} Train Accuracy: {2} Test Accuracy{3}".format(
                    step, train_loss, train_accuracy, test_accuracy
                )


        save_path = saver.save(sess, 'tictactoe.ckpt')

Ausführung lernen

2017-07-04 14:19:57.084696: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-07-04 14:19:57.084722: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-07-04 14:19:57.084728: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-07-04 14:19:57.084733: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Iteration: 0 Loss: 1.08315229416 Train Accuracy: 0.425999999046 Test Accuracy0.426455020905
Iteration: 100 Loss: 0.748883843422 Train Accuracy: 0.814249992371 Test Accuracy0.785185158253
Iteration: 200 Loss: 0.629662871361 Train Accuracy: 0.934000015259 Test Accuracy0.894179880619
Iteration: 300 Loss: 0.600810408592 Train Accuracy: 0.960250020027 Test Accuracy0.914285719395
Iteration: 400 Loss: 0.594145357609 Train Accuracy: 0.964749991894 Test Accuracy0.913227498531
Iteration: 500 Loss: 0.582351207733 Train Accuracy: 0.975499987602 Test Accuracy0.925396800041
Iteration: 600 Loss: 0.575868725777 Train Accuracy: 0.981500029564 Test Accuracy0.920634925365
Iteration: 700 Loss: 0.571496605873 Train Accuracy: 0.98425000906 Test Accuracy0.924867749214
Iteration: 800 Loss: 0.571447372437 Train Accuracy: 0.98474997282 Test Accuracy0.919576704502
Iteration: 900 Loss: 0.567611455917 Train Accuracy: 0.98575001955 Test Accuracy0.925396800041
Iteration: 1000 Loss: 0.567007541656 Train Accuracy: 0.98575001955 Test Accuracy0.925396800041
Iteration: 1100 Loss: 0.566512107849 Train Accuracy: 0.986000001431 Test Accuracy0.928571403027
Iteration: 1200 Loss: 0.566121637821 Train Accuracy: 0.986000001431 Test Accuracy0.925925910473
Iteration: 1300 Loss: 0.565603733063 Train Accuracy: 0.986500024796 Test Accuracy0.924338638783
Iteration: 1400 Loss: 0.56520396471 Train Accuracy: 0.986750006676 Test Accuracy0.925396800041
Iteration: 1500 Loss: 0.564830541611 Train Accuracy: 0.986999988556 Test Accuracy0.926455020905
Iteration: 1600 Loss: 0.564735352993 Train Accuracy: 0.986999988556 Test Accuracy0.926455020905
Iteration: 1700 Loss: 0.564707398415 Train Accuracy: 0.986999988556 Test Accuracy0.926984131336
Iteration: 1800 Loss: 0.56460750103 Train Accuracy: 0.986999988556 Test Accuracy0.9280423522
Iteration: 1900 Loss: 0.564545154572 Train Accuracy: 0.986999988556 Test Accuracy0.926455020905
Iteration: 2000 Loss: 0.564533174038 Train Accuracy: 0.986999988556 Test Accuracy0.928571403027
Iteration: 2100 Loss: 0.564481317997 Train Accuracy: 0.986999988556 Test Accuracy0.927513241768
Iteration: 2200 Loss: 0.564553022385 Train Accuracy: 0.986999988556 Test Accuracy0.92962962389
Iteration: 2300 Loss: 0.583365738392 Train Accuracy: 0.96850001812 Test Accuracy0.919576704502
Iteration: 2400 Loss: 0.566257119179 Train Accuracy: 0.986249983311 Test Accuracy0.926455020905
Iteration: 2500 Loss: 0.563695311546 Train Accuracy: 0.987999975681 Test Accuracy0.926455020905
Iteration: 2600 Loss: 0.563434004784 Train Accuracy: 0.987999975681 Test Accuracy0.92962962389
Iteration: 2700 Loss: 0.563206732273 Train Accuracy: 0.988250017166 Test Accuracy0.9280423522
Iteration: 2800 Loss: 0.563172519207 Train Accuracy: 0.988250017166 Test Accuracy0.931746006012
Iteration: 2900 Loss: 0.563154757023 Train Accuracy: 0.988250017166 Test Accuracy0.931746006012
Iteration: 3000 Loss: 0.563151359558 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 3100 Loss: 0.563149094582 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 3200 Loss: 0.563141226768 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 3300 Loss: 0.563139140606 Train Accuracy: 0.988250017166 Test Accuracy0.931216955185
Iteration: 3400 Loss: 0.563138246536 Train Accuracy: 0.988250017166 Test Accuracy0.931216955185
Iteration: 3500 Loss: 0.563154280186 Train Accuracy: 0.988250017166 Test Accuracy0.931746006012
Iteration: 3600 Loss: 0.563149809837 Train Accuracy: 0.988250017166 Test Accuracy0.931216955185
Iteration: 3700 Loss: 0.563176214695 Train Accuracy: 0.988250017166 Test Accuracy0.929100513458
Iteration: 3800 Loss: 0.563181519508 Train Accuracy: 0.988250017166 Test Accuracy0.931216955185
Iteration: 3900 Loss: 0.563153684139 Train Accuracy: 0.988250017166 Test Accuracy0.929100513458
Iteration: 4000 Loss: 0.563127815723 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 4100 Loss: 0.563163101673 Train Accuracy: 0.988250017166 Test Accuracy0.9280423522
Iteration: 4200 Loss: 0.563137412071 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 4300 Loss: 0.563160598278 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 4400 Loss: 0.563147187233 Train Accuracy: 0.988250017166 Test Accuracy0.926984131336
Iteration: 4500 Loss: 0.563141047955 Train Accuracy: 0.988250017166 Test Accuracy0.9280423522
Iteration: 4600 Loss: 0.563162863255 Train Accuracy: 0.988250017166 Test Accuracy0.930158734322
Iteration: 4700 Loss: 0.563196718693 Train Accuracy: 0.988250017166 Test Accuracy0.929100513458
Iteration: 4800 Loss: 0.563158690929 Train Accuracy: 0.988250017166 Test Accuracy0.92962962389
Iteration: 4900 Loss: 0.563124537468 Train Accuracy: 0.988250017166 Test Accuracy0.926984131336
Iteration: 5000 Loss: 0.563167691231 Train Accuracy: 0.988250017166 Test Accuracy0.930158734322
Iteration: 5100 Loss: 0.563187777996 Train Accuracy: 0.988250017166 Test Accuracy0.930687844753
Iteration: 5200 Loss: 0.56315112114 Train Accuracy: 0.988250017166 Test Accuracy0.9280423522
Iteration: 5300 Loss: 0.570619702339 Train Accuracy: 0.981000006199 Test Accuracy0.924867749214
Iteration: 5400 Loss: 0.576466858387 Train Accuracy: 0.976249992847 Test Accuracy0.927513241768
Iteration: 5500 Loss: 0.563514411449 Train Accuracy: 0.988250017166 Test Accuracy0.928571403027
Iteration: 5600 Loss: 0.562488973141 Train Accuracy: 0.989000022411 Test Accuracy0.93439155817
Iteration: 5700 Loss: 0.56244301796 Train Accuracy: 0.989000022411 Test Accuracy0.933333337307
Iteration: 5800 Loss: 0.562356352806 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 5900 Loss: 0.562411308289 Train Accuracy: 0.989000022411 Test Accuracy0.935978829861
Iteration: 6000 Loss: 0.562405765057 Train Accuracy: 0.989000022411 Test Accuracy0.933862447739
Iteration: 6100 Loss: 0.562349438667 Train Accuracy: 0.989000022411 Test Accuracy0.935449719429
Iteration: 6200 Loss: 0.562380075455 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 6300 Loss: 0.562388420105 Train Accuracy: 0.989000022411 Test Accuracy0.935449719429
Iteration: 6400 Loss: 0.562395453453 Train Accuracy: 0.989000022411 Test Accuracy0.935449719429
Iteration: 6500 Loss: 0.562419772148 Train Accuracy: 0.989000022411 Test Accuracy0.933862447739
Iteration: 6600 Loss: 0.562360167503 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 6700 Loss: 0.562407493591 Train Accuracy: 0.989000022411 Test Accuracy0.93439155817
Iteration: 6800 Loss: 0.562382221222 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 6900 Loss: 0.562420666218 Train Accuracy: 0.989000022411 Test Accuracy0.932804226875
Iteration: 7000 Loss: 0.562407851219 Train Accuracy: 0.989000022411 Test Accuracy0.933862447739
Iteration: 7100 Loss: 0.562392890453 Train Accuracy: 0.989000022411 Test Accuracy0.932804226875
Iteration: 7200 Loss: 0.562432050705 Train Accuracy: 0.989000022411 Test Accuracy0.932804226875
Iteration: 7300 Loss: 0.562389314175 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 7400 Loss: 0.562418997288 Train Accuracy: 0.989000022411 Test Accuracy0.933333337307
Iteration: 7500 Loss: 0.562441766262 Train Accuracy: 0.989000022411 Test Accuracy0.935449719429
Iteration: 7600 Loss: 0.562380254269 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 7700 Loss: 0.562415003777 Train Accuracy: 0.989000022411 Test Accuracy0.935449719429
Iteration: 7800 Loss: 0.562358081341 Train Accuracy: 0.989000022411 Test Accuracy0.93439155817
Iteration: 7900 Loss: 0.562432765961 Train Accuracy: 0.989000022411 Test Accuracy0.935978829861
Iteration: 8000 Loss: 0.562436521053 Train Accuracy: 0.989000022411 Test Accuracy0.936507940292
Iteration: 8100 Loss: 0.562419176102 Train Accuracy: 0.989000022411 Test Accuracy0.93439155817
Iteration: 8200 Loss: 0.562465846539 Train Accuracy: 0.989000022411 Test Accuracy0.931746006012
Iteration: 8300 Loss: 0.562432646751 Train Accuracy: 0.989000022411 Test Accuracy0.934920608997
Iteration: 8400 Loss: 0.562426924706 Train Accuracy: 0.989000022411 Test Accuracy0.933333337307
Iteration: 8500 Loss: 0.562418758869 Train Accuracy: 0.989000022411 Test Accuracy0.933862447739
Iteration: 8600 Loss: 0.562417984009 Train Accuracy: 0.989000022411 Test Accuracy0.935978829861
Iteration: 8700 Loss: 0.562437176704 Train Accuracy: 0.989000022411 Test Accuracy0.935978829861
Iteration: 8800 Loss: 0.578755617142 Train Accuracy: 0.972500026226 Test Accuracy0.927513241768
Iteration: 8900 Loss: 0.565938591957 Train Accuracy: 0.986000001431 Test Accuracy0.935449719429
Iteration: 9000 Loss: 0.562196016312 Train Accuracy: 0.989250004292 Test Accuracy0.935978829861
Iteration: 9100 Loss: 0.561437726021 Train Accuracy: 0.990000009537 Test Accuracy0.938624322414
Iteration: 9200 Loss: 0.561364352703 Train Accuracy: 0.990000009537 Test Accuracy0.938624322414
Iteration: 9300 Loss: 0.561371803284 Train Accuracy: 0.990000009537 Test Accuracy0.938624322414
Iteration: 9400 Loss: 0.561358273029 Train Accuracy: 0.990000009537 Test Accuracy0.939682543278
Iteration: 9500 Loss: 0.561344504356 Train Accuracy: 0.990000009537 Test Accuracy0.939153432846
Iteration: 9600 Loss: 0.561368823051 Train Accuracy: 0.990000009537 Test Accuracy0.939682543278
Iteration: 9700 Loss: 0.56139343977 Train Accuracy: 0.990000009537 Test Accuracy0.937566161156
Iteration: 9800 Loss: 0.561371207237 Train Accuracy: 0.990000009537 Test Accuracy0.939682543278
Iteration: 9900 Loss: 0.561351060867 Train Accuracy: 0.990000009537 Test Accuracy0.939682543278

Über die generierte Datei

http://qiita.com/neriai/items/791e6f4dd8d08775542b

Lernergebnis

Lernmodell

FireShot Capture 37 - TensorBoard - http___localhost_6006_#graphs.png

Über Tensor Board

http://qiita.com/neriai/items/a7b47127462ecf0fcc1d

Grafik lernen

FireShot Capture 39 - TensorBoard - http___localhost_6006_#scalars.png

Zusammenfassung

Vielleicht ist es die am einfachsten zu verstehende Modellstruktur. ――Ist es auf Überlernen zurückzuführen, dass die Obergrenze des Diagramms subtil ist? ――Es scheint schwierig zu sein, Trainingsdaten zu erstellen. ――Nächstes Mal würde ich es gerne tatsächlich spielen.

[PYTHON] Ich habe ein ○ ✕ Spiel mit TensorFlow gemacht