[PYTHON] [TensorFlow] [Keras] Neural network construction with Keras

In this article, "Build a neural network with Python without using a library --Qiita" An article (Neural network construction with chainer --Qiita) that a senior of the company tried with Chainer I also tried it with Keras (material for LT in the company)

Keras Documentation

• Caution This article was written in the Keras 1.x era. It may not work in 2.x because it has not been verified.

All sources and execution results (Jupyter Notebook)

The source and execution result are listed in the following Gist. Building a neural network with Keras

Implementation and description

Data creation

Create input data. This is exactly the same implementation.

Input data (cum-teacher data) creation

import numpy as np
import sklearn.datasets
import matplotlib
import matplotlib.pyplot as plt

np.random.seed(0)
X,y=sklearn.datasets.make_moons(200,noise=0.20)
plt.scatter(X[:,0], X[:,1], s=40, c=y, cmap=plt.cm.Spectral)

Modeling

Create a model in Keras.

Modeling

import keras
from keras.models import Sequential
from keras.layers import Dense, Activation

model = Sequential()
model.add(Dense(output_dim=6, input_dim=2))
model.add(Activation('tanh'))
model.add(Dense(output_dim=2))
model.compile(optimizer='Adam', loss='mse')

It's about the same as I ported Chainer's. (However, it seems that the loss function is hidden in Classifier in Chainer, but what is used?) => [Addition] In the following article, it was said that softmax_cross_entropy is used in Chainer's Classifier. Notes on changes in Chainer 1.5 --studylog / North Cloud

Sequential Model Guide-Keras Documentation Activation function --Keras Documentation Optimization-Keras Documentation Objective Function-Keras Documentation

(Epoch was 20000 in Chainer, but it was mistakenly changed to 2000 at the time of transplantation, and it was a poor result if the hidden layer was 3 which is the same as Chainer. When the hidden layer was set to 6, it was 2000. I got a good result)

Learning

Train with model.fit (). Regarding y_train, it is converted into a two-dimensional array (vector) with a probability of 0 and a probability of 1. (Chainer seems to convert it without permission)

x_train = X
y_train = keras.utils.np_utils.to_categorical(y, nb_classes=2)
model.fit(x=x_train, y=y_train, nb_epoch=2000)

Numpy Utility --Keras Documentation

Result display

Result output

# https://gist.github.com/dennybritz/ff8e7c2954dd47a4ce5f
def plot_decision_boundary(pred_func):
    # Set min and max values and give it some padding
    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
    h = 0.01
    # Generate a grid of points with distance h between them
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
    # Predict the function value for the whole gid
    Z = pred_func(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    # Plot the contour and training examples
    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)
    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Spectral)

def predict(model, x_data):
    y = model.predict(x_data)
    return np.argmax(y.data, axis=1) #Get the index that is the maximum value

plot_decision_boundary(lambda x: predict(model, x))

I think this gives almost the same result as the original article.