[PYTHON] Try implementing XOR with Keras Functional API

Introduction

I decided to start Keras in earnest, so I wrote the code to build the prediction model of XOR. I'd like to create a complex model such as Graph Convolutional Networks in the future, so I'll start with the Functional API.

Source

Like this. Without Batch Normalization, it was easy to fall into a local solution, so I added it. In many cases, the final layer was a linear layer, but it was a 0, 1 classification problem and I was not convinced, so I chose the sigmoid function. The number of units in the two intermediate layers is set to eight.

sample.py

import tensorflow as tf
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Dense, Input, Dropout, BatchNormalization

import numpy as np


def main():
    x_input = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
    y_input = np.array([[0], [1], [1], [0]])

    input_tensor = Input(shape=(x_input.shape[1]))
    # x = Dense(units=32, activation="tanh", kernel_initializer='random_normal')(input_tensor)
    x = Dense(units=8, activation="relu", kernel_initializer='random_normal', use_bias=True)(input_tensor)
    x = BatchNormalization()(x)
    x = Dropout(0.1)(x)
    x = Dense(units=8, activation="relu", kernel_initializer='random_normal', use_bias=True)(x)
kernel_initializer='random_normal', use_bias=False)(x)
    output_layer = Dense(units=1, activation='sigmoid', kernel_initializer='random_normal', use_bias=False)(x)
    model = Model(input_tensor, output_layer)

    model.compile(loss='mse',  optimizer='sgd', metrics=['accuracy'])
    model.summary()

    #Learning
    model.fit(x_input, y_input, nb_epoch=2000, batch_size=2, verbose=2)

    #Forecast
    print(model.predict(np.array([[0, 0]])))
    print(model.predict(np.array([[1, 0]])))
    print(model.predict(np.array([[0, 1]])))
    print(model.predict(np.array([[1, 1]])))


if __name__ == "__main__":
    main()

Model overview

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 2)]               0
_________________________________________________________________
dense (Dense)                (None, 8)                 24
_________________________________________________________________
batch_normalization (BatchNo (None, 8)                 32
_________________________________________________________________
dropout (Dropout)            (None, 8)                 0
_________________________________________________________________
dense_1 (Dense)              (None, 8)                 72
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 8
=================================================================
Total params: 136
Trainable params: 120
Non-trainable params: 16

Prediction result

[[0.09092495]]
[[0.9356866]]
[[0.90092343]]
[[0.08152929]]

in conclusion

I tried it as a starting point this time, but I would like to try various things such as parameter tuning and visualization.