Notes for using TensorFlow on Bash on Ubuntu on Windows So, I can use TensorFlow, but I don't know how to use it.
There is a library called TFLearn that makes it easier to use TensorFlow, so I included that as well.
$ pip install tflearn
Looking at the sample code, there was a program called logical.py that seems to learn logical operations, so I tried it this time.
In logical.py, learning of multiple logical operations is grouped together, so I extracted only the relevant parts.
import tensorflow as tf
import tflearn
# Logical OR operator
X = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
Y = [[0.], [1.], [1.], [1.]]
# Graph definition
with tf.Graph().as_default():
g = tflearn.input_data(shape=[None, 2])
g = tflearn.fully_connected(g, 128, activation='linear')
g = tflearn.fully_connected(g, 128, activation='linear')
g = tflearn.fully_connected(g, 1, activation='sigmoid')
g = tflearn.regression(g, optimizer='sgd', learning_rate=2.,
loss='mean_square')
# Model training
m = tflearn.DNN(g)
m.fit(X, Y, n_epoch=100, snapshot_epoch=False)
# Test model
print("Testing OR operator")
print("0 or 0:", m.predict([[0., 0.]]))
print("0 or 1:", m.predict([[0., 1.]]))
print("1 or 0:", m.predict([[1., 0.]]))
print("1 or 1:", m.predict([[1., 1.]]))
It's my first look, but somehow I understand the meaning. When this was done, the result was as follows.
--
Training Step: 100 | total loss: 0.00227
| SGD | epoch: 100 | loss: 0.00227 -- iter: 4/4
--
Testing OR operator
0 or 0: [[0.031054211780428886]]
0 or 1: [[0.9823662638664246]]
1 or 0: [[0.9786670207977295]]
1 or 1: [[0.9999874830245972]]
If you look at it digitally as 0 or 1, you have learned OR.
By the way, in the first code, 128 intermediate layers are connected by two layers. I tried to delete the middle layer because it would not be necessary for the middle layer like learning OR. Instead, I increased the number of learnings to 2000.
import tensorflow as tf
import tflearn
# Logical OR operator
X = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
Y = [[0.], [1.], [1.], [1.]]
# Graph definition
with tf.Graph().as_default():
g = tflearn.input_data(shape=[None, 2])
g = tflearn.fully_connected(g, 1, activation='sigmoid')
g = tflearn.regression(g, optimizer='sgd', learning_rate=2., loss='mean_square')
# Model training
m = tflearn.DNN(g)
m.fit(X, Y, n_epoch=2000, snapshot_epoch=False)
# Test model
print("Testing OR operator")
print("0 or 0:", m.predict([[0., 0.]]))
print("0 or 1:", m.predict([[0., 1.]]))
print("1 or 0:", m.predict([[1., 0.]]))
print("1 or 1:", m.predict([[1., 1.]]))
The result of doing this.
--
Training Step: 2000 | total loss: 0.00098
| SGD | epoch: 2000 | loss: 0.00098 -- iter: 4/4
--
Testing OR operator
0 or 0: [[0.041201911866664886]]
0 or 1: [[0.9756871461868286]]
1 or 0: [[0.9764388799667358]]
1 or 1: [[0.9999741315841675]]
It seems that OR learning was done properly.
Then I tried learning AND. The code is the same as just changing the teacher signal to AND.
import tensorflow as tf
import tflearn
# Logical AND operator
X = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
Y = [[0.], [0.], [0.], [1.]]
# Graph definition
with tf.Graph().as_default():
g = tflearn.input_data(shape=[None, 2])
g = tflearn.fully_connected(g, 1, activation='sigmoid')
g = tflearn.regression(g, optimizer='sgd', learning_rate=2., loss='mean_square')
# Model training
m = tflearn.DNN(g)
m.fit(X, Y, n_epoch=2000, snapshot_epoch=False)
# Test model
print("Testing AND operator")
print("0 and 0:", m.predict([[0., 0.]]))
print("0 and 1:", m.predict([[0., 1.]]))
print("1 and 0:", m.predict([[1., 0.]]))
print("1 and 1:", m.predict([[1., 1.]]))
The result of doing this.
--
Training Step: 2000 | total loss: 0.00137
| SGD | epoch: 2000 | loss: 0.00137 -- iter: 4/4
--
Testing AND operator
0 and 0: [[8.591794176027179e-05]]
0 and 1: [[0.04014528915286064]]
1 and 0: [[0.03964542970061302]]
1 and 1: [[0.9525935053825378]]
Certainly it became AND.
Since OR and AND can be linearly separated, an intermediate layer is unnecessary, but XOR cannot be linearly separated, so an intermediate layer is required. However, in the sample code, instead of learning XOR directly, NAND and OR are trained and combined and used.
I didn't know why I didn't train it directly, so I wrote a code to train XOR directly.
import tensorflow as tf
import tflearn
# Logical XOR operator
X = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
Y = [[0.], [1.], [1.], [0.]]
# Graph definition
with tf.Graph().as_default():
g = tflearn.input_data(shape=[None, 2])
g = tflearn.fully_connected(g, 2, activation='sigmoid')
g = tflearn.fully_connected(g, 1, activation='sigmoid')
g = tflearn.regression(g, optimizer='sgd', learning_rate=2., loss='mean_square')
# Model training
m = tflearn.DNN(g)
m.fit(X, Y, n_epoch=2000, snapshot_epoch=False)
# Test model
print("Testing XOR operator")
print("0 xor 0:", m.predict([[0., 0.]]))
print("0 xor 1:", m.predict([[0., 1.]]))
print("1 xor 0:", m.predict([[1., 0.]]))
print("1 xor 1:", m.predict([[1., 1.]]))
This just made the teacher signal XOR and added two intermediate layers. So, when I did this, it turned out like this.
--
Training Step: 2000 | total loss: 0.25000
| SGD | epoch: 2000 | loss: 0.25000 -- iter: 4/4
--
Testing XOR operator
0 xor 0: [[0.5000224709510803]]
0 xor 1: [[0.5000009536743164]]
1 xor 0: [[0.49999910593032837]]
1 xor 1: [[0.4999775290489197]]
I can't learn that. So, when I googled, the next page was caught. It's the usual Stack Overflow site.
tflearn / tensorflow does not learn xor
According to this, with the standard setting, the initial value of the weight seems to be quite narrow with a standard deviation of 0.02. Therefore, it seems better to widen the range of initial weight values from -1 to 1.
import tensorflow as tf
import tflearn
# Logical XOR operator
X = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
Y = [[0.], [1.], [1.], [0.]]
# Graph definition
with tf.Graph().as_default():
tnorm = tflearn.initializations.uniform(minval=-1.0, maxval=1.0)
g = tflearn.input_data(shape=[None, 2])
g = tflearn.fully_connected(g, 2, activation='sigmoid', weights_init=tnorm)
g = tflearn.fully_connected(g, 1, activation='sigmoid', weights_init=tnorm)
g = tflearn.regression(g, optimizer='sgd', learning_rate=2., loss='mean_square')
# Model training
m = tflearn.DNN(g)
m.fit(X, Y, n_epoch=2000, snapshot_epoch=False)
# Test model
print("Testing XOR operator")
print("0 xor 0:", m.predict([[0., 0.]]))
print("0 xor 1:", m.predict([[0., 1.]]))
print("1 xor 0:", m.predict([[1., 0.]]))
print("1 xor 1:", m.predict([[1., 1.]]))
The result of executing by changing the setting of the initial value of the weight in this way.
--
Training Step: 2000 | total loss: 0.00131
| SGD | epoch: 2000 | loss: 0.00131 -- iter: 4/4
--
Testing XOR operator
0 xor 0: [[0.03527239337563515]]
0 xor 1: [[0.9663047790527344]]
1 xor 0: [[0.9607295393943787]]
1 xor 1: [[0.03082425333559513]]
I was able to learn XOR safely.
It is easy to understand because the code corresponds to the idea of neural networks. It may not be possible to fine-tune TensorFlow, but for those who don't know what TensorFlow can do in the first place, why not start with TF Learn?
Recommended Posts