This article is an easy-to-understand output of ** Deep Learning from scratch Chapter 6 Error back propagation method **. I was able to understand it myself in the humanities, so I hope you can read it comfortably. Also, I would be more than happy if you could refer to it when studying this book.
The Affine layer is a layer in which the input signal of the neuron is multiplied by the weight, summed, and biased.
class Affine:#Layer implementation of processing that weights the input signal and adds its sum and bias
def __init__(self, W, b):
self.W = W
self.b = b
self.x = None
self.dW = None #Create an instance variable to save each derivative obtained by backpropagation processing
self.db = None #Create an instance variable to save each derivative obtained by backpropagation processing
def forward(self, x):
self.x = x
out = np.dot(x, self.W) + self.b
return out
def backward(self, dout):
dx = np.dot(dout, self.W.T)
self.dW = np.dot(self.x.T, dout)
self.db = np.sum(dout, axis=0)#To support multiple data (batch)
#Find the derivative of the bias by the sum of the previous derivatives
return dx
Since the back propagation process uses multiplication and addition in the forward propagation process of the Affine layer, the derivative of the bias inherits the previous derivative, and the derivative of the weight and the input value is transposed and replaced with the previous derivative. It is calculated by multiplying by. Bias is not just inherited, but batch processing is done as above.
Recommended Posts