In this article, I will write a continuation of the previous article "Python beginners touch Pytorch (1)".
y = 3x \\
In the above function, the slope is "3" </ font>. This is because if you increase x by "1" in a two-dimensional plane, y increases by "3".
The slope can be said to be the amount of change in the graph. It will be easier to understand if you can imagine the graph you learned in elementary school.
\\
y = 3x\\
y = x^2+3x+4
The above functions are all one-variable functions. So what is a function with two or more variables?
y = x_1+2x_2+3x_3
The above function is a two-variable function. The big difference from a one-variable function is the number of variables. There are two and three variables called x in the function.
For functions with two or more variables, consider the slope for each variable. Therefore,
x_Slope about 1= 1\\x_Slope about 2= 2\\x_Slope about 3= 3
It looks like.
You can see that the range of expression of the function has expanded due to the increase in the types of slopes. (The width of expression is the number of variations of the answer "y".)
A neural network uses a lot of variables to express a function with a complicated shape and functions as a discriminator.
I will omit the detailed relationship between functions and variables. If you are interested, we recommend that you study hard with books and reliable sites.
y = x^2 + x\\
\\
y' = 2x + 1
y'means derivative. Differentiating the above equation gives the following equation. I will explain the basic principle. The rules of differentiation when x is a variable are described below.
y = ax^n \hspace{5mm} \rightarrow \hspace{5mm} y' = nax^{n-1} \\
y = x \hspace{5mm} \rightarrow \hspace{5mm} y' = 1
This time, if you can understand this, there is no problem.
This is really the foundation of the foundation in differentiation. Differentiations that cannot be introduced here, such as complicated differentiation methods, trigonometric functions, and logarithms, are here
Link: Beautiful story of high school mathematics Differential formula list (from basic to development)
Also, for quadratic functions and above, you can check the slope focusing on a certain point of the function.
Example) \hspace{1mm} y=x^2 + 3 \hspace{5mm} \rightarrow \hspace{5mm} y' = 2x \hspace{2mm}(differential)\\
x=Slope at 1\hspace{3mm}y` =2 ・ 1=2\\
x=Slope at 2\hspace{3mm}y` =2 ・ 2=4
If you can understand the above, it is easy to understand this article.
y = 3x
Is differentiated by Pytorch and the slope 3 is calculated.
python
import torch
x = torch.ones(3,3,requires_grad=True)
a = torch.tensor([[3,3,3],[3,3,3],[3,3,3]])
y = a*x
When dealing with differentiation, add "requires_grad = True" when declaring the variable you want to differentiate. This is the code that allows differentiation. Furthermore, if the content of the variable used for differentiation is an integer type, an error will occur. Declare it as a floating point type. </ font>
result
tensor([[3., 3., 3.],
[3., 3., 3.],
[3., 3., 3.]], grad_fn=<MulBackward0>)
Make sure that the statement "grad_fn =" is added to the output.
Here are some points to note when differentiating. In Pytorch, when differentiating, must be one-dimensional. Since it is </ font>, all the tensors are put together in one.
python
y = torch.sum(y)
torch.sum () was introduced in the previous article. Please refer to the previous article. Python beginners touch Pytorch (1)
We will finally differentiate.
python
y.backward()
print(x.grad)
.backward () is a necessary preparation when performing differentiation. Also, when doing .backward (), an error will occur if the variable being differentiated is not one-dimensional.
Let's see the result
result
tensor([[3., 3., 3.],
[3., 3., 3.],
[3., 3., 3.]])
The differential calculation was performed firmly, and the result was that the slope was "3".
out = x + 2y + 3z\\
\\
Slope with respect to x= 1\\
Slope about y= 2\\
Slope with respect to z= 3
I will calculate with Pytorch.
python
x = torch.ones((3,3),requires_grad=True)
y = torch.ones((3,3),requires_grad=True)
z = torch.ones((3,3),requires_grad=True)
out = 1*x+2*y+3*z
out = torch.sum(out)
out.backward()
print('Slope with respect to x')
print(x.grad)
print('Slope about y')
print(y.grad)
print('Slope with respect to z')
print(z.grad)
result
Slope with respect to x
tensor([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]])
Slope about y
tensor([[2., 2., 2.],
[2., 2., 2.],
[2., 2., 2.]])
Slope with respect to z
tensor([[3., 3., 3.],
[3., 3., 3.],
[3., 3., 3.]])
\hspace{1mm} y=x^2 + 3 \hspace{5mm} \rightarrow \hspace{5mm} y' = 2x \hspace{2mm}(differential)\\
x=Slope at 1\hspace{3mm}y` =2 ・ 1=2\\
x=Slope at 2\hspace{3mm}y` =2 ・ 2=4
We will calculate this with Pytorch.
python
a = torch.ones(3,3)
x1 = torch.ones((3,3),requires_grad=True)
x2 = torch.tensor([[2.0,2.0,2.0],
[2.0,2.0,2.0],
[2.0,2.0,2.0]],requires_grad=True)
b = torch.tensor([[3,3,3],
[3,3,3],
[3,3,3]])
out1 = a*x1*x1+b
out2 = a*x2*x2+b
out1 = torch.sum(out1)
out2 = torch.sum(out2)
out1.backward()
print("x =Slope at 1")
print(x1.grad)
out2.backward()
print("x =Slope at 2")
print(x2.grad)
result
x =Slope at 1
tensor([[2., 2., 2.],
[2., 2., 2.],
[2., 2., 2.]])
x =Slope at 2
tensor([[4., 4., 4.],
[4., 4., 4.],
[4., 4., 4.]])
You can calculate it well.
Recommended Posts