[PYTHON] Notify Slack of how Keras is learning

Recently, I often train models with Python's deep learning library Keras. Deep learning generally takes a long time to train the model, and I don't always log in to the GPU (or similar powerful) server to check the console, so I would appreciate it if you could report the progress each time. is.

So, from Keras' callback function, hit Slack's WebAPI to try to notify by chat.

Get Slack WebAPI Key

First, get the required API key. The task of getting an API key for Slack is just a summary, as there are a lot of resources out there and the official documentation is kind.

From Slack Official, press the "Start Building" button at the top of the screen. Then, the name of the app that uses the API key (appropriate and OK, for example, "notifier") and the pop-up that registers the team that wants to use the API key will appear. Enter it as appropriate.

Once the app is registered, we want to post to Slack externally this time, so enable Incoming Webhooks.

After that, if you follow the instructions on the screen, the API key will be issued.

When the API key is issued, the sample code will be displayed, so it is a good idea to copy and paste it to check the operation once.

Keras callback settings

Keras provides a class called LambdaCallback that makes it easy to generate custom callbacks, so you can just use it:

import os

from keras.callbacks import LambdaCallback

hostname = os.uname()[1]

callbacks = []

slack_command = 'curl -X POST -H \'Content-type: application/json\' --data \'{{"text":"Here is {}.\nepoch:{:03d}, loss:{:.7f}, val_loss:{:.7f}"}}\' https://hooks.slack.com/services/<your_key_here>'
slack_callback = LambdaCallback(
    on_epoch_end=lambda epoch, logs: os.system(slack_command.format(hostname, epoch, logs['loss'], logs['val_loss'])))
callbacks.append(slack_callback)

#All you have to do is pass a Callback when calling the Keras model training method as usual.
history = model.fit_generator(train_generator, samples_per_epoch, nb_epoch, callbacks=callbacks,
                    validation_data=val_generator, nb_val_samples=nb_val_samples)

Replace <your_key_here> with your API key. Also, loss and val_loss exist without anything, but if you want other metrics, you can pass them to the metrics argument of themodel.compile ()method and name themLambdaCallback.You can also refer to it. For registration of custom metrics, refer to Applicable page of Keras official document.

As you can see from the code above, LambdaCallback has hooks such as ʻon_epoch_begin and ʻon_batch_end in addition to ʻon_epoch_end`. However, it would be annoying if Slack was notified on a batch-by-batch basis, so I might not use it for this purpose. (It may be convenient for logging to CSV etc.)

Reference link

• Slack API: https://api.slack.com/ --Keras Official (Callbacks): https://keras.io/callbacks/ --Getting the host name in Python: http://qiita.com/iorionda/items/cfceac93c9e87d430077