[PYTHON] PyTorch learning memo (I made the same model as Karas)
Introduction
I had thought that I wouldn't study PyTorch soon, but I had a lot of time, so I studied with a book (PyTorch Neural Network Implementation Handbook) in one hand. The prerequisite knowledge of the author at the time of writing this article is
- TensorFlow and Keras are known normally
- PyTorch is a complete beginner
is. I think there are other people who are similar, so I hope you find it helpful.
Rough impression of PyTorch
After all, it's only half a day since I started studying, so it may be wrong, but as a rough impression, I thought that the implementation complexity was just halfway between Keras and TensorFlow. The definition of the model is almost the same as Keras. On the contrary, the learning part gave me the impression that I had to write a fair amount of bare code. However, it wasn't that hard because it worked just by copying the code that was working elsewhere. On the flip side, Keras, which can hide this area cleanly, may be wonderful.
Method of approach
I don't use my head to copy the code of the book, so I decided to create a model with exactly the same structure and algorithm as the one I implemented in Keras for the classification of CIFAR-10 that I had tried in the past. .. There were some things I noticed for the first time when I implemented it from scratch, so I think it was an efficient study method.
Implementation code and environment
The implementation code explained below is uploaded on Github below. As anyone who has done it will know, CIFAR-10 is a fairly difficult example, and if you make a model normally, the accuracy is only about 60%. I think that the model introduced here is certainly the structure of the highly accurate model that appeared in Kaggle. (Kaggle also did other data pre-processing, but that's omitted). The accuracy is different every time, but if it is in good condition, it will go up to about 80%.
https://github.com/makaishi2/sample-data/blob/master/notebooks/cifar10_keras.ipynb
https://github.com/makaishi2/sample-data/blob/master/notebooks/cifar10_pytorch.ipynb
I used Google Colab as the environment. The Keras / PyTorch relationship did not require any additional installation. I think that only the Japanese localization module of matplotlib was additionally introduced. As expected, Google Colab is very convenient in such cases.
Code commentary
With this as a preface, let's get into the actual code description.
Data loading
As for the training data of CIFAR-10, like Keras, PyTorch has a function and I could load it immediately. Since it's a big deal, I'll compare the code for this part as well.
Keras
#Import Keras library
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation
#Import of other libraries
!pip install japanize_matplotlib | tail -n 1
import matplotlib.pyplot as plt
import japanize_matplotlib
import numpy as np
#Read training data
from keras.datasets import cifar10
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
classes = ('plane', 'car', 'bird', 'cat',
'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
#Calculation of the number of classes to be classified
class_labels_count = len(set(y_train.flatten()))
# One Hot Encoding
from keras.utils import np_utils
y_train_ohe = np_utils.to_categorical(y_train, class_labels_count)
y_test_ohe = np_utils.to_categorical(y_test, class_labels_count)
PyTorch
#PyTooch related library import
import torch
import torchvision
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torchvision.transforms as transforms
#Other library import
%matplotlib inline
import numpy as np
!pip install japanize_matplotlib | tail -n 1
import matplotlib.pyplot as plt
import japanize_matplotlib
#Number of classification classes
num_classes = 10
#Number of learning repetitions
nb_epoch = 20
#How many images to use in one learning
batch_size = 128
#Read training data
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
download=True, transform=transform)
train_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2)
testset = torchvision.datasets.CIFAR10(root='./data', train=False,download=True, transform=transform)
test_loader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=2)
classes = ('plane', 'car', 'bird', 'cat',
'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
PyTorch was different from Keras in terms of how it handles training data.
Classes specialized for learning called DataSet and DataLoader have been created, and they will be used.
A DataSet is a tuple of input data and a set of correct label values, and is prepared as an Iterator. The input data is a PyTorch-specific variable of the class Tensor.
The specification of batch_size used in the so-called "** mini-batch learning method **" is done in the DataLoader (I didn't notice this at all at first).
Another point is that there is a difference in the correct label data. In the case of Keras, the correct answer data must be One Hot Encoding. We implement it at the end of the code. In PyTorch, it seems that the encoding of the label value is done inside the framework, and the value without encoding (value such as 6 or 3) can be used as it is as the correct answer value at the time of learning. PyTorch seems to be more convenient in this regard.
Image display of input data
It has nothing to do with the learning itself, but since it's a big deal, let's display the first 10 of the read learning data as an image. The result is as follows.
Even such a simple thing can be implemented quite differently with Keras / PyTorch due to the difference in how to hold the data.
Keras
plt.figure(figsize=(15, 4))
for i in range(10):
ax = plt.subplot(1, 10, i + 1)
image = x_train[i]
label = y_train[i][0]
plt.imshow(image)
ax.set_title(classes[label], fontsize=16)
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()
PyTorch
plt.figure(figsize=(15, 4))
for i in range(10):
ax = plt.subplot(1, 10, i + 1)
image, label = trainset[i]
np_image = image.numpy().copy()
img = np.transpose(np_image, (1, 2, 0))
img2 = (img + 1)/2
plt.imshow(img2)
ax.set_title(classes[label], fontsize=16)
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()
The first feature of PyTorch is that it loops through a variable called trainset and extracts a set of (image, label) one by one.
Another difference is that it requires a lot of processing to display the image data on the screen. In the case of Keras, the data was originally a numpy array, and it could be displayed by passing it to the plt.imshow function as it is, but in the case of PyTorch, the following three steps of processing were required.
- Since the image variable is a variable of Tensor class, convert it to Numpy
- Since the order of the axes is different, replace them with the
np.transposefunction. - Since the range of training data is [-1, 1], change it to the range of [0, 1].
I didn't notice the last story, but when I understood it, it seems that the data is obtained by processing with the transforms.Normalize function used when reading the data.
Well, if you have the state before processing with another variable, you should be able to implement it more efficiently, but since I was a beginner of PyTorch and did not know what to do specifically, I am OK with this once.
Model generation
The next step is to generate the most essential model. Regarding this, I have the impression that Keras / PyTorch is almost the same.
Keras
def cnn_model(x_train, class_labels_count):
model = Sequential()
model.add(Conv2D(32, (3, 3), padding="same", input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(class_labels_count))
model.add(Activation('softmax'))
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy']
)
return model
#Model generation
model = cnn_model(x_train, class_labels_count)
PyTorch
#Model class definition
class cifar10_cnn(nn.Module):
def __init__(self, num_classes):
super(cifar10_cnn,self).__init__()
self.conv1 = nn.Conv2d(3, 32, 3, padding=(1,1), padding_mode='replicate')
self.conv2 = nn.Conv2d(32, 32, 3)
self.conv3 = nn.Conv2d(32, 64, 3, padding=(1,1), padding_mode='replicate')
self.conv4 = nn.Conv2d(64, 64, 3)
self.relu = nn.ReLU(inplace=True)
self.dropout1 = nn.Dropout(0.25)
self.dropout2 = nn.Dropout(0.5)
self.maxpool = nn.MaxPool2d((2,2))
self.classifier1 = nn.Linear(2304, 512)
self.classifier2 = nn.Linear(512, num_classes)
self.features = nn.Sequential(
self.conv1,
self.relu,
self.conv2,
self.relu,
self.maxpool,
self.dropout1,
self.conv3,
self.relu,
self.conv4,
self.relu,
self.dropout1,
self.maxpool)
self.classifier = nn.Sequential(
self.classifier1,
self.relu,
self.dropout2,
self.classifier2)
def forward(self, x):
x = self.features(x)
x = x.view(x.size(0), -1)
x = self.classifier(x)
return x
#Check GPU
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(device)
#Model instance generation and GPU allocation
net = cifar10_cnn(num_classes).to(device)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(net.parameters())
In Keras, you can define a model with a function (you can do it solidly without using a function), but in PyTorch, it seems to be a good idea to define a class. However, I get the impression that they are similar in terms of the amount of implementation code.
There is one point that I had a hard time with PyTorch, and it seems that I can not simply specify padding ='same' like Keras regarding padding. When I googled, I found the following article in the article on qiita, so I used it.
https://qiita.com/syoyo/items/ddff3268b4dfa3ebb3d6
Also, since PyTorch can dynamically declare the structure of the model, I think that the magic number 2304 of nn.Linear (2304, 512) can be calculated, but prioritize moving it first. I proceeded.
In PyTorch, if you want to use GPU, it seems that you need to explicitly assign the device like net = cifar10_cnn (num_classes) .to (device). In Keras, if you have a GPU, you can use it without permission, so Keras was more convenient in that respect.
Model structure confirmation
Let's check if this really has the same structure. Here is the command and its output to check the structure with Keras / PyTorch.
Keras
#Model summary view
model.summary()
result
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 32, 32, 32) 896
_________________________________________________________________
activation (Activation) (None, 32, 32, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 30, 30, 32) 9248
_________________________________________________________________
activation_1 (Activation) (None, 30, 30, 32) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 15, 15, 32) 0
_________________________________________________________________
dropout (Dropout) (None, 15, 15, 32) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 15, 15, 64) 18496
_________________________________________________________________
activation_2 (Activation) (None, 15, 15, 64) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 13, 13, 64) 36928
_________________________________________________________________
activation_3 (Activation) (None, 13, 13, 64) 0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 6, 64) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 6, 6, 64) 0
_________________________________________________________________
flatten (Flatten) (None, 2304) 0
_________________________________________________________________
dense (Dense) (None, 512) 1180160
_________________________________________________________________
activation_4 (Activation) (None, 512) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 512) 0
_________________________________________________________________
dense_1 (Dense) (None, 10) 5130
_________________________________________________________________
activation_5 (Activation) (None, 10) 0
=================================================================
Total params: 1,250,858
Trainable params: 1,250,858
Non-trainable params: 0
_________________________________________________________________
PyTorch
#Model summary view
from torchsummary import summary
summary(net,(3,32,32))
result
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Conv2d-1 [-1, 32, 32, 32] 896
Conv2d-2 [-1, 32, 32, 32] 896
ReLU-3 [-1, 32, 32, 32] 0
ReLU-4 [-1, 32, 32, 32] 0
ReLU-5 [-1, 32, 32, 32] 0
Conv2d-6 [-1, 32, 30, 30] 9,248
Conv2d-7 [-1, 32, 30, 30] 9,248
ReLU-8 [-1, 32, 30, 30] 0
ReLU-9 [-1, 32, 30, 30] 0
ReLU-10 [-1, 32, 30, 30] 0
MaxPool2d-11 [-1, 32, 15, 15] 0
MaxPool2d-12 [-1, 32, 15, 15] 0
Dropout-13 [-1, 32, 15, 15] 0
Dropout-14 [-1, 32, 15, 15] 0
Conv2d-15 [-1, 64, 15, 15] 18,496
Conv2d-16 [-1, 64, 15, 15] 18,496
ReLU-17 [-1, 64, 15, 15] 0
ReLU-18 [-1, 64, 15, 15] 0
ReLU-19 [-1, 64, 15, 15] 0
Conv2d-20 [-1, 64, 13, 13] 36,928
Conv2d-21 [-1, 64, 13, 13] 36,928
ReLU-22 [-1, 64, 13, 13] 0
ReLU-23 [-1, 64, 13, 13] 0
ReLU-24 [-1, 64, 13, 13] 0
Dropout-25 [-1, 64, 13, 13] 0
Dropout-26 [-1, 64, 13, 13] 0
MaxPool2d-27 [-1, 64, 6, 6] 0
MaxPool2d-28 [-1, 64, 6, 6] 0
Linear-29 [-1, 512] 1,180,160
Linear-30 [-1, 512] 1,180,160
ReLU-31 [-1, 512] 0
ReLU-32 [-1, 512] 0
ReLU-33 [-1, 512] 0
Dropout-34 [-1, 512] 0
Dropout-35 [-1, 512] 0
Linear-36 [-1, 10] 5,130
Linear-37 [-1, 10] 5,130
================================================================
Total params: 2,501,716
Trainable params: 2,501,716
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.01
Forward/backward pass size (MB): 3.76
Params size (MB): 9.54
Estimated Total Size (MB): 13.31
----------------------------------------------------------------
The number of elements fits perfectly in all layers and looks nice.
In the case of PyTorch, if you execute the variable net which is an instance of the model as it is, the following information will also be displayed.
net
cifar10_cnn(
(conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
(conv2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1))
(conv3): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
(conv4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1))
(relu): ReLU(inplace=True)
(dropout1): Dropout(p=0.25, inplace=False)
(dropout2): Dropout(p=0.5, inplace=False)
(maxpool): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
(classifier1): Linear(in_features=2304, out_features=512, bias=True)
(classifier2): Linear(in_features=512, out_features=10, bias=True)
(features): Sequential(
(0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
(1): ReLU(inplace=True)
(2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1))
(3): ReLU(inplace=True)
(4): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
(5): Dropout(p=0.25, inplace=False)
(6): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1))
(9): ReLU(inplace=True)
(10): Dropout(p=0.25, inplace=False)
(11): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
)
(classifier): Sequential(
(0): Linear(in_features=2304, out_features=512, bias=True)
(1): ReLU(inplace=True)
(2): Dropout(p=0.5, inplace=False)
(3): Linear(in_features=512, out_features=10, bias=True)
)
)
Learning
Now everything is ready. We will finally carry out learning. As I mentioned at the beginning, this part is (currently) overwhelmingly easy for Keras.
Keras
#Number of learning repetitions
nb_epoch = 20
#How many images to use in one learning
batch_size = 128
#Learning
history = model.fit(
x_train, y_train_ohe, batch_size=batch_size, epochs=nb_epoch, verbose=1,
validation_data=(x_test, y_test_ohe), shuffle=True
)
PyTorch
Learning
train_loss_list = []
train_acc_list = []
val_loss_list = []
val_acc_list = []
for epoch in range(nb_epoch):
train_loss = 0
train_acc = 0
val_loss = 0
val_acc = 0
#train
net.train()
for i, (images, labels) in enumerate(train_loader):
#view()Do not convert with
images, labels = images.to(device), labels.to(device)
optimizer.zero_grad()
outputs = net(images)
loss = criterion(outputs, labels)
train_loss += loss.item()
train_acc += (outputs.max(1)[1] == labels).sum().item()
loss.backward()
optimizer.step()
avg_train_loss = train_loss / len(train_loader.dataset)
avg_train_acc = train_acc / len(train_loader.dataset)
#val
net.eval()
with torch.no_grad():
for images, labels in test_loader:
#view()Do not convert with
images = images.to(device)
labels = labels.to(device)
outputs = net(images)
loss = criterion(outputs, labels)
val_loss += loss.item()
val_acc += (outputs.max(1)[1] == labels).sum().item()
avg_val_loss = val_loss / len(test_loader.dataset)
avg_val_acc = val_acc / len(test_loader.dataset)
print ('Epoch [{}/{}], loss: {loss:.4f} val_loss: {val_loss:.4f}, val_acc: {val_acc:.4f}'
.format(epoch+1, nb_epoch, i+1, loss=avg_train_loss, val_loss=avg_val_loss, val_acc=avg_val_acc))
train_loss_list.append(avg_train_loss)
train_acc_list.append(avg_train_acc)
val_loss_list.append(avg_val_loss)
val_acc_list.append(avg_val_acc)
The execution result of each is as follows. However, Keras is the output of the fit function prepared by the framework itself, while PyTorch only outputs the output similar to it by hand-set the print function, so it is not correct to compare in the first place. I can say.
Keras
Epoch 1/20
391/391 [==============================] - 7s 11ms/step - loss: 4.8051 - accuracy: 0.2355 - val_loss: 1.5342 - val_accuracy: 0.4480
Epoch 2/20
391/391 [==============================] - 4s 10ms/step - loss: 1.4853 - accuracy: 0.4612 - val_loss: 1.2996 - val_accuracy: 0.5420
Epoch 3/20
391/391 [==============================] - 4s 10ms/step - loss: 1.3215 - accuracy: 0.5309 - val_loss: 1.1627 - val_accuracy: 0.5996
Epoch 4/20
391/391 [==============================] - 4s 10ms/step - loss: 1.2014 - accuracy: 0.5732 - val_loss: 1.0446 - val_accuracy: 0.6388
Epoch 5/20
391/391 [==============================] - 4s 10ms/step - loss: 1.1124 - accuracy: 0.6070 - val_loss: 0.9813 - val_accuracy: 0.6627
Epoch 6/20
391/391 [==============================] - 4s 10ms/step - loss: 1.0317 - accuracy: 0.6355 - val_loss: 0.9245 - val_accuracy: 0.6772
Epoch 7/20
391/391 [==============================] - 4s 10ms/step - loss: 0.9625 - accuracy: 0.6639 - val_loss: 0.8732 - val_accuracy: 0.7022
Epoch 8/20
391/391 [==============================] - 4s 10ms/step - loss: 0.9309 - accuracy: 0.6748 - val_loss: 0.8433 - val_accuracy: 0.7064
Epoch 9/20
391/391 [==============================] - 4s 10ms/step - loss: 0.8730 - accuracy: 0.6922 - val_loss: 0.8197 - val_accuracy: 0.7174
Epoch 10/20
391/391 [==============================] - 4s 10ms/step - loss: 0.8575 - accuracy: 0.6982 - val_loss: 0.7722 - val_accuracy: 0.7310
Epoch 11/20
391/391 [==============================] - 4s 10ms/step - loss: 0.8030 - accuracy: 0.7178 - val_loss: 0.7756 - val_accuracy: 0.7272
Epoch 12/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7841 - accuracy: 0.7244 - val_loss: 0.7372 - val_accuracy: 0.7468
Epoch 13/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7384 - accuracy: 0.7429 - val_loss: 0.7738 - val_accuracy: 0.7340
Epoch 14/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7321 - accuracy: 0.7462 - val_loss: 0.7177 - val_accuracy: 0.7501
Epoch 15/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6976 - accuracy: 0.7530 - val_loss: 0.7478 - val_accuracy: 0.7477
Epoch 16/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7047 - accuracy: 0.7537 - val_loss: 0.7160 - val_accuracy: 0.7608
Epoch 17/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6638 - accuracy: 0.7682 - val_loss: 0.7111 - val_accuracy: 0.7630
Epoch 18/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6509 - accuracy: 0.7709 - val_loss: 0.7099 - val_accuracy: 0.7610
Epoch 19/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6346 - accuracy: 0.7770 - val_loss: 0.6933 - val_accuracy: 0.7691
Epoch 20/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6119 - accuracy: 0.7833 - val_loss: 0.7006 - val_accuracy: 0.7619
PyTorch
Epoch [1/20], loss: 0.0124 val_loss: 0.0104, val_acc: 0.5187
Epoch [2/20], loss: 0.0094 val_loss: 0.0081, val_acc: 0.6432
Epoch [3/20], loss: 0.0079 val_loss: 0.0075, val_acc: 0.6791
Epoch [4/20], loss: 0.0069 val_loss: 0.0066, val_acc: 0.7173
Epoch [5/20], loss: 0.0062 val_loss: 0.0060, val_acc: 0.7405
Epoch [6/20], loss: 0.0058 val_loss: 0.0057, val_acc: 0.7557
Epoch [7/20], loss: 0.0053 val_loss: 0.0054, val_acc: 0.7666
Epoch [8/20], loss: 0.0050 val_loss: 0.0054, val_acc: 0.7656
Epoch [9/20], loss: 0.0047 val_loss: 0.0052, val_acc: 0.7740
Epoch [10/20], loss: 0.0045 val_loss: 0.0052, val_acc: 0.7708
Epoch [11/20], loss: 0.0042 val_loss: 0.0051, val_acc: 0.7790
Epoch [12/20], loss: 0.0040 val_loss: 0.0051, val_acc: 0.7797
Epoch [13/20], loss: 0.0038 val_loss: 0.0051, val_acc: 0.7802
Epoch [14/20], loss: 0.0037 val_loss: 0.0050, val_acc: 0.7812
Epoch [15/20], loss: 0.0035 val_loss: 0.0049, val_acc: 0.7910
Epoch [16/20], loss: 0.0034 val_loss: 0.0049, val_acc: 0.7840
Epoch [17/20], loss: 0.0033 val_loss: 0.0049, val_acc: 0.7936
Epoch [18/20], loss: 0.0031 val_loss: 0.0049, val_acc: 0.7917
Epoch [19/20], loss: 0.0030 val_loss: 0.0048, val_acc: 0.8008
Epoch [20/20], loss: 0.0029 val_loss: 0.0050, val_acc: 0.7900
Learning curve
Finally, let's take a look at the learning curve for each case. As you know, in deep learning, the accuracy is different each time because it uses random numbers. Please note that it is not possible to argue which relaim work is better based on this result alone.
Keras
#Learning curve(Loss function value)
plt.figure(figsize=(8,6))
plt.plot(history.history['val_loss'],label='adam', lw=3, c='b')
plt.title('Learning curve(Loss function value)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()
#Learning curve(accuracy)
plt.figure(figsize=(8,6))
plt.plot(history.history['val_accuracy'],label='adam', lw=3, c='b')
plt.title('Learning curve(accuracy)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()
PyTorch
#Learning curve(Loss function value)
plt.figure(figsize=(8,6))
plt.plot(val_loss_list,label='adam', lw=3, c='b')
plt.title('Learning curve(Loss function value)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()
#Learning curve(accuracy)
plt.figure(figsize=(8,6))
plt.plot(val_acc_list,label='adam', lw=3, c='b')
plt.title('Learning curve(accuracy)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()
References
PyTorch Neural Network Implementation Handbook Keiichiro Miyamoto (Author), Yohei Okawa (Author), Takuya Mouri (Author) Shuwa System
Recommended Posts