Summarize the error back propagation of the sigmoid function
The derivative of the sigmoi degree function has a beautiful and simple form.
** sigmoid function: **
sigmoid(x) = \frac{1}{1+e^{-x}}
** Differentiation of sigmoid function: **
sigmoid'(x) = \frac{1}{1+e^{-x}} * ( 1 - \frac{1}{1+e^{-x}})
Therefore, the error back propagation code of the sigmoid function is also simplified as follows.
Class Sigmoid(object):
def __init__(self, x):
self.x = x
def forward(self):
y = 1.0 / (1.0 + np.exp(- self.x))
self.y = y
return y
def backward(self, grad_to_y):
grad_to_x = grad_to_y * self.y * (1.0 - self.y)
return grad_to_x
Recommended Posts