[PYTHON] Manabu NN with app: Understand tensor flow playground with Keras
This article was submitted on October 13, 2020, but it was deleted and slightly rewritten, and it was submitted again on October 14, 2020 at 15:12.
TensorFlow Playground ([Neural Network Playground](https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=4,2&seed=0.40707&showTestData=false&discretize= false & percTrainData = 50 & x = true & y = true & xTimesY = false & xSquared = false & ySquared = false & cosX = false & sinX = false & cosY = false & sinY = false & collectStats = false & problem = classification & initZero = false & hideText = false)) is a neural network learning application that runs on the browser. With this, anyone can easily understand neural networks in a short period of time.
Please refer to Manabu NN: Moving Artificial Intelligence-Tensor Flow Playground.
However, in order to understand the contents in more detail, it is one way to try to make the same thing in another programming language. In this article, we will do this using keras.
Circle
The image above is the initial screen of the Tensor Flow Playground. Circles are selected for the output data and X1 and X2 are selected for the input data. I will try it.
To get an idea of exactly what this is doing, it's a good idea to try the same move in Keras. So I made the following program.
#Initialization
%matplotlib inline
import pandas_datareader.data as web
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import datetime
import keras
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import SGD
from keras import models
from keras import layers
from keras import regularizers
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
Data construction
n=250
theta=np.random.uniform(0, 360, n)
x1=np.sin(theta)*np.random.uniform(3,5, n)
x2=np.cos(theta)*np.random.uniform(3,5, n)
y=np.ones(n)
plt.figure(figsize=(4, 4), dpi=50)
plt.scatter(x1,x2)
theta=np.random.uniform(0, 360, n)
x11=np.sin(theta)*np.random.uniform(-0, 2, n)
x22=np.cos(theta)*np.random.uniform(-0,2, n)
yy=np.zeros(n)
plt.scatter(x11,x22)
Execution result
#Creation of training data and test data
x1=np.concatenate([x1,x11],axis=0)
x2=np.concatenate([x2,x22],axis=0)
y=np.concatenate([y,yy],axis=0)
X=np.stack([x1,x2],1)#.reshape(-1,2)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.5, random_state=42)
Network construction and optimization
model = models.Sequential()
model.add(layers.Dense(8, activation='tanh', input_shape=(2,)))
model.add(layers.Dense(8, activation='tanh'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(
X_train,
y_train,
batch_size=10,
epochs=500,
verbose=0)
#Evaluation of results
results = model.evaluate(X_train, y_train)
results
# 8/8 [==============================] - 0s 1ms/step - loss: 3.0139e-05 - accuracy: 1.0000
# [3.0139368391246535e-05, 1.0]
results = model.evaluate(X_test, y_test)
results
# 8/8 [==============================] - 0s 2ms/step - loss: 1.6788e-04 - accuracy: 1.0000
# [0.0001678814151091501, 1.0]
Both results have zero error. All the questions were correct. Let's change the input data for a better understanding.
I was able to classify the data neatly with a simple perceptron. Let's do this with Keras.
X2=np.stack([x1**2,x2**2],1)#.reshape(-1,2)
X2_train, X2_test, y_train, y_test = train_test_split(
X2, y, test_size=0.33, random_state=42)
model = models.Sequential()
model.add(layers.Dense(1, activation='tanh', input_shape=(2,)))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(
X2_train,
y_train,
batch_size=10,
epochs=500,
verbose=0)
The result is as follows.
results = model.evaluate(X2_train, y_train)
results
# 11/11 [==============================] - 0s 3ms/step - loss: 8.3843e-08 - accuracy: 1.0000
# [8.38432754335372e-08, 1.0]
results = model.evaluate(X2_test, y_test)
results
# 6/6 [==============================] - 0s 2ms/step - loss: 8.3747e-08 - accuracy: 1.0000
# [8.374658477805497e-08, 1.0]
In both cases, the accuracy rate is 100% and the error is zero.
normal distribution
Next is the normal distribution.
First, build the data.
n=250
x1=np.random.normal(2,0.5, n)
x2=np.random.normal(2,0.5, n)
y=np.ones(n)
plt.figure(figsize=(4, 4), dpi=50)
plt.scatter(x1,x2)
x11=np.random.normal(-2,0.5, n)
x22=np.random.normal(-2,0.5, n)
yy=np.zeros(n)
plt.scatter(x11,x22)
x1=np.concatenate([x1,x11],axis=0)
x2=np.concatenate([x2,x22],axis=0)
y=np.concatenate([y,yy],axis=0)
X=np.stack([x1,x2],1)#.reshape(-1,2)
The neural network uses the same one as for the circle. The result is
results = model.evaluate(X_train, y_train)
results
# 8/8 [==============================] - 0s 1ms/step - loss: 3.0139e-05 - accuracy: 1.0000
# [3.0139368391246535e-05, 1.0]
The correct answer rate with zero error is 100% again.
In this case
It will also be.
XOR
Next is XOR.
n=125
x1=np.random.uniform(0,5, n)
x2=np.random.uniform(0,5, n)
y=np.ones(n)
x11=np.random.uniform(-5,0, n)
x22=np.random.uniform(-5,0, n)
yy=np.ones(n)
x1=np.concatenate([x1,x11],axis=0)
x2=np.concatenate([x2,x22],axis=0)
y=np.concatenate([y,yy],axis=0)
plt.figure(figsize=(4, 4), dpi=50)
plt.scatter(x1,x2)
x11=np.random.uniform(0,5, n)
x22=np.random.uniform(0,-5, n)
yy=np.zeros(n)
x111=np.random.uniform(-5,0, n)
x222=np.random.uniform(0,5, n)
yyy=np.zeros(n)
x11=np.concatenate([x11,x111],axis=0)
x22=np.concatenate([x22,x222],axis=0)
yy=np.concatenate([yy,yyy],axis=0)
plt.scatter(x11,x22)
len(x1),len(x2)
# (250, 250)
You can see that 250 pieces of data have been created.
Shuffle and divide the training data and test data.
x1=np.concatenate([x1,x11],axis=0)
x2=np.concatenate([x2,x22],axis=0)
y=np.concatenate([y,yy],axis=0)
X=np.stack([x1,x2],1)#.reshape(-1,2)
len(y)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.5, random_state=42)
We will try to classify this data by NN of 2 layers (1st layer 4 units, 2nd layer 2 units). Result is
results = model.evaluate(X_train, y_train)
results
# 8/8 [==============================] - 0s 2ms/step - loss: 0.0818 - accuracy: 0.9640
# [0.08181502670049667, 0.9639999866485596]
results = model.evaluate(X_test, y_test)
results
# 8/8 [==============================] - 0s 2ms/step - loss: 0.1269 - accuracy: 0.9520
# [0.1269153356552124, 0.9520000219345093]
The error is almost zero, and the correct answer rate is over 90%.
Next, let's change the structure of the data as follows.
X_train, X_test, y_train, y_test = train_test_split(
x1*x2, y, test_size=0.33, random_state=42)
model = models.Sequential()
model.add(layers.Dense(1, activation='tanh', input_shape=(1,)))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(
X_train,
y_train,
batch_size=10,
epochs=500,
verbose=0)
The result is as follows.
results = model.evaluate(X_train, y_train)
results
# 11/11 [==============================] - 0s 1ms/step - loss: 5.6793e-04 - accuracy: 1.0000
# [0.0005679333116859198, 1.0]
results = model.evaluate(X_test, y_test)
results
# 6/6 [==============================] - 0s 3ms/step - loss: 0.0070 - accuracy: 0.9939
# [0.007003221195191145, 0.9939393997192383]
The error was almost zero, and the correct answer rate was almost 100%.
reference:
"Simulator de Manabu Neural Network" (Amazon Kindle Publishing)
"Unraveling the brain with brain, mind, and artificial intelligence mathematics" (Blue Bucks) "Approach to sparse modeling based on compressed sensing"
Neural network that anyone can understand: I tried regularization on the tensor flow playground