keras is a wrapper library of ** Theano / Tensorflow **, and by using ** keras **, less code than writing solid in Theano / Tensorflow It can be described by quantity. Regarding keras, there is a page that has already been explained, so I will give it to that, and I investigated how to use Tensorboard from keras, so I would like to explain it briefly.

When outputting the log for ** Tensorboard **, use the mechanism called ** Callback **. The registered ** Callback ** will be called at a fixed timing. The timing of being called depends on the type of ** Callback **, but ** Callback ** for ** Tensorboard ** is called ** just before learning ** and ** at the end of each epoch **. Will be done.

Just before learning, ** tf.merge_all_summaries ** and ** tf.train.SummaryWriter ** are called. At the end of each epoch, ** add_summary ** is called and a log is output. You can also change the frequency of calling callbacks if each epoch is too much.

** Callback ** for ** Tensorflow ** is created with ** keras.callbacks.TensorBoard **. Pass the created Callback to fit. In ** histogram_freq **, specify the frequency to output the histogram data of ** Tensorboard **. If histogram_freq = 1, epoch data will be output every time.

`python`


    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, histogram_freq=1)
    cbks = [tb_cb]
            
    history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch = nb_epoch, verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))

If the network structure changes between ** Training ** and ** Test ** like ** Dropout **, the following description is required before building the Model. If you do not describe it, you will die when building the Model.

`python`


    KTF.set_learning_phase(1)

** learning_phase ** is used when ** Training ** and ** Test ** behave differently. If the network does not change between ** Training ** and ** Test **, it will not be entered.

** learning_phase ** is set to ** 1 ** during ** Training ** and ** 0 ** during ** Test **. This value switches the network. You don't have to do anything because ** keras ** automatically inputs the value. For example, ** 0 ** is set for ** Validation **.

Simple example

A simple ** Maltilayer Neural Network ** example looks like this:

`python`


import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.optimizers import SGD
from keras.utils import np_utils
import keras.callbacks
import keras.backend.tensorflow_backend as KTF
import tensorflow as tf

batch_size = 128
nb_classes = 10
nb_epoch   = 20
nb_data    = 28*28
log_filepath = './log'

# load data
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# reshape
X_train = X_train.reshape(X_train.shape[0], X_train.shape[1]*X_train.shape[2])
X_test = X_test.reshape(X_test.shape[0], X_test.shape[1]*X_test.shape[2])

# rescale
X_train = X_train.astype(np.float32)
X_test  = X_test.astype(np.float32)
X_train /= 255
X_test  /= 255

# convert class vectors to binary class matrices (one hot vectors)
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

old_session = KTF.get_session()

with tf.Graph().as_default():
    session = tf.Session('')
    KTF.set_session(session)
    KTF.set_learning_phase(1)
    # build model
    model = Sequential()
    model.add(Dense(512, input_shape=(nb_data,), init='normal',name='dense1'))
    model.add(Activation('relu', name='relu1'))
    model.add(Dropout(0.2, name='dropout1'))
    model.add(Dense(512, init='normal', name='dense2'))
    model.add(Activation('relu', name='relu2'))
    model.add(Dropout(0.2, name='dropout2'))
    model.add(Dense(10, init='normal', name='dense3'))
    model.add(Activation('softmax', name='softmax1'))       
    model.summary()

    model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.01), metrics=['accuracy'])

    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, histogram_freq=1)
    cbks = [tb_cb]
            
    history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch = nb_epoch, verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))

    score = model.evaluate(X_test, Y_test, verbose=0)
    print('Test score:', score[0])
    print('Test accuracy;', score[1])

KTF.set_session(old_session)

Doing this will print the ** Keras ** runtime log to standard output.

Layer (type)                       Output Shape        Param #     Connected to                     
====================================================================================================
dense1 (Dense)                     (None, 512)         401920      dense_input_1[0][0]              
____________________________________________________________________________________________________
relu1 (Activation)                 (None, 512)         0           dense1[0][0]                     
____________________________________________________________________________________________________
dropout1 (Dropout)                 (None, 512)         0           relu1[0][0]                      
____________________________________________________________________________________________________
dense2 (Dense)                     (None, 512)         262656      dropout1[0][0]                   
____________________________________________________________________________________________________
relu2 (Activation)                 (None, 512)         0           dense2[0][0]                     
____________________________________________________________________________________________________
dropout2 (Dropout)                 (None, 512)         0           relu2[0][0]                      
____________________________________________________________________________________________________
dense3 (Dense)                     (None, 10)          5130        dropout2[0][0]                   
____________________________________________________________________________________________________
softmax1 (Activation)              (None, 10)          0           dense3[0][0]                     
====================================================================================================
Total params: 669706
____________________________________________________________________________________________________
Train on 60000 samples, validate on 10000 samples
Epoch 1/20
60000/60000 [==============================] - 5s - loss: 1.2522 - acc: 0.6646 - val_loss: 0.5603 - val_acc: 0.8695
Epoch 2/20
60000/60000 [==============================] - 5s - loss: 0.5533 - acc: 0.8445 - val_loss: 0.3862 - val_acc: 0.8962
Epoch 3/20
60000/60000 [==============================] - 5s - loss: 0.4381 - acc: 0.8728 - val_loss: 0.3291 - val_acc: 0.9084
Epoch 4/20
60000/60000 [==============================] - 5s - loss: 0.3879 - acc: 0.8867 - val_loss: 0.2963 - val_acc: 0.9147
Epoch 5/20
60000/60000 [==============================] - 6s - loss: 0.3536 - acc: 0.8975 - val_loss: 0.2753 - val_acc: 0.9198
Epoch 6/20
60000/60000 [==============================] - 5s - loss: 0.3271 - acc: 0.9047 - val_loss: 0.2575 - val_acc: 0.9268
Epoch 7/20
60000/60000 [==============================] - 5s - loss: 0.3059 - acc: 0.9113 - val_loss: 0.2421 - val_acc: 0.9301
Epoch 8/20
60000/60000 [==============================] - 5s - loss: 0.2873 - acc: 0.9168 - val_loss: 0.2302 - val_acc: 0.9331
Epoch 9/20
60000/60000 [==============================] - 5s - loss: 0.2751 - acc: 0.9199 - val_loss: 0.2198 - val_acc: 0.9359
Epoch 10/20
60000/60000 [==============================] - 5s - loss: 0.2612 - acc: 0.9246 - val_loss: 0.2092 - val_acc: 0.9386
Epoch 11/20
60000/60000 [==============================] - 5s - loss: 0.2507 - acc: 0.9274 - val_loss: 0.1998 - val_acc: 0.9416
Epoch 12/20
60000/60000 [==============================] - 6s - loss: 0.2383 - acc: 0.9311 - val_loss: 0.1930 - val_acc: 0.9437
Epoch 13/20
60000/60000 [==============================] - 5s - loss: 0.2288 - acc: 0.9340 - val_loss: 0.1859 - val_acc: 0.9461
Epoch 14/20
60000/60000 [==============================] - 5s - loss: 0.2195 - acc: 0.9367 - val_loss: 0.1776 - val_acc: 0.9492
Epoch 15/20
60000/60000 [==============================] - 5s - loss: 0.2107 - acc: 0.9391 - val_loss: 0.1715 - val_acc: 0.9511
Epoch 16/20
60000/60000 [==============================] - 5s - loss: 0.2040 - acc: 0.9405 - val_loss: 0.1658 - val_acc: 0.9514
Epoch 17/20
60000/60000 [==============================] - 5s - loss: 0.1969 - acc: 0.9423 - val_loss: 0.1607 - val_acc: 0.9533
Epoch 18/20
60000/60000 [==============================] - 5s - loss: 0.1922 - acc: 0.9442 - val_loss: 0.1559 - val_acc: 0.9554
Epoch 19/20
60000/60000 [==============================] - 5s - loss: 0.1853 - acc: 0.9454 - val_loss: 0.1518 - val_acc: 0.9558
Epoch 20/20
60000/60000 [==============================] - 6s - loss: 0.1813 - acc: 0.9470 - val_loss: 0.1472 - val_acc: 0.9568
('Test score:', 0.1472099754229188)
('Test accuracy;', 0.95679999999999998)

Looking at the log output by ** Tensorboard **, it looks like the following.

The histograms of ** Weight **, ** Bias **, and ** Activation ** are output as shown below. The Histogram Tag is a bit like that, it seems that the name is not used for now.

Graph looks like this.

I was able to easily output the ** Tensorboard ** log from ** Keras **. ** Keras ** has just been released and there is still room for improvement, but it seems to be updated daily. The description is cleaner than writing ** Tensorflow ** in solid, so I think it's very convenient when you want to give it a try.

[PYTHON] [TF] How to use Tensorboard from Keras

python

python

Simple example

python

`python`

`python`

`python`