[PYTHON] DeepRunning ~ Level2, Level3.1 ~
Level2. JDLA certification program "Deep learning course that can be crushed in the field in 3 months"
I applied for a JDLA-certified course to take the E qualification. I will write the result of learning for each unit.
・ Applied Mathematics "Level 3. Linear Algebra" "Level 4. Probability / Statistics" "Level 5. Information Theory"
Level3. Applied mathematics
3-1. Linear algebra
・ Learning goals
(1) Check how to find the eigenvalues and eigenvectors. (2) Deepen understanding of eigenvalue decomposition. (3) Get an overview of singular values and singular vectors. (4) Get an overview of singular value decomposition.
3-1-1. Scalars and vectors
【scalar】 Those that can perform four arithmetic operations (+-× ÷) with ordinary numbers It becomes a coefficient with respect to the vector and can be multiplied by a scalar. 【vector】 It has "size" and "direction". It is mainly illustrated by arrows and displays scalars as a set.
3-1-2. Matrix
【queue】
・ Scalar is turned upside down ・ Enter $ \ vec {A} $ with → in a sequence of vectors. -As an example, write as follows.
\begin{aligned}\begin{pmatrix}
3 & 2 & 0 \\
0 & 2 & 0 \\
0 & 0 & 1
\end{pmatrix}\end{aligned}
[Example of matrix product]
\begin{aligned}\begin{pmatrix}
2 & 1 \\
4 & 1
\end{pmatrix}\begin{pmatrix}
1 & 3 \\
3 & 1
\end{pmatrix}=\begin{pmatrix}
2\times 1+1\times 3 & 2\times 3+1\times 1 \\
4\times 1+1\times 3 & 4\times 3+1\times 1
\end{pmatrix}\\
=\begin{pmatrix}
5 & 7 \\
7 & 13
\end{pmatrix}\end{aligned}
3-1-3. Identity matrix and inverse matrix
[Identity matrix]
I=\begin{pmatrix}
1 & & \\
& 1 & \\
& & \ldots
\end{pmatrix}
[Matrix and inverse matrix product] Multiplying a matrix by the inverse matrix yields an identity matrix. The "-1" on the right shoulder of the inverse matrix is called inverse.
AA^{-1}=A^{-1}A=I
3-1-4. How to find the inverse matrix
In the form of a matrix, multiply the i-th row by c, add the c-fold of the t-th row to the s-th row, and swap the p-th and q-th rows ...
Finding the inverse matrix by the row basic transformation is called the sweeping method. (= Gauss sweeping method)
It seems that it is required by the operation of the sweep method inside programming, It's easier to find it by calculation.
\begin{aligned}\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}^{-1}=\dfrac {1}{ad-bc}\begin{pmatrix}
d & -b \\
-c & a
\end{pmatrix}\end{aligned}
3-1-5. Eigenvalues and eigenvectors
For a certain matrix A, there is a special vector $ \ vec {x} $ and a coefficient λ such that the following equation holds.
A\overrightarrow {x}=\lambda \overrightarrow {x}
The product of the matrix A and the special vector $ \ vec {x} $ is It is the same as the product of λ, which is just the number of scalars, and the special vector $ \ vec {x} $.
The special vector $ \ vec {x} $ is called the ** eigenvector **, and its coefficient λ is called the ** eigenvalue **.
Specific examples are the following eigenvalues and eigenvectors. The eigenvector is a ratio, which may be 2 or 3 in addition to 1.
\begin{pmatrix}
1 & 4 \\
2 & 3
\end{pmatrix}\begin{pmatrix}
1 \\
1
\end{pmatrix}=5\begin{pmatrix}
1 \\
1
\end{pmatrix}
3-1-6. How to find eigenvalues and eigenvectors
\begin{vmatrix}
3-\lambda & 2 & 0 \\
0 & 2-\lambda & 0 \\
0 & 0 & 1-\lambda
\end{vmatrix}=0
(3 - λ)(2 - λ)(1 - λ) = 0 Therefore, λ = 3 or 2 or 1
\begin{pmatrix}
3-\lambda & 2 & 0 \\
0 & 2-\lambda & 0 \\
0 & 0 & 1-\lambda
\end{pmatrix}\begin{pmatrix}
x_1 \\
x_2 \\
x_3
\end{pmatrix}=0
Substitute λ = 3 or 2 or 1 for the solution, or
\begin{pmatrix}
3 & 2 & 0 \\
0 & 2 & 0 \\
0 & 0 & 1
\end{pmatrix}\begin{pmatrix}
x_1 \\
x_2 \\
x_3
\end{pmatrix}=λ\begin{pmatrix}
x_1 \\
x_2 \\
x_3
\end{pmatrix}
Substitute λ = 3 or 2 or 1 for and solve.
λ =At the time of 3\overrightarrow {x}=\begin{pmatrix}1 \\ 0 \\ 0\end{pmatrix}Constant times\\
λ =At the time of 2\overrightarrow {x}=\begin{pmatrix}2 \\ -1 \\ 0\end{pmatrix}Constant times\\
λ =At the time of 1\overrightarrow {x}=\begin{pmatrix}0 \\ 0 \\ 1\end{pmatrix}Constant times
Will be.
3-1-7. Eigenvalue decomposition
- Must be a square matrix.
Suppose the matrix A has eigenvalues $ λ_1, λ_2, λ_3, ... $ and eigenvectors $ ν_1, ν_2, ν_3, ... $. Matrix Λ (lambda) in which these eigenvalues are arranged diagonally
Λ=\begin{pmatrix}
λ_1 & & \\
& λ_2 & \\
& & \ldots
\end{pmatrix}
And the matrix in which the corresponding eigenvectors are arranged
V=\begin{pmatrix}
ν_1 & ν_2 &･ ･ ･
\end{pmatrix}
When, it is associated with AV = VA.
Therefore, it can be transformed into $ A = VΛV ^ {-1} $.
Converting a square matrix into the product of three matrices is called ** eigenvalue decomposition **.
⇒ When finding the power of a matrix, $ VΛV ^ {-1} × VΛV ^ {-1} V ^ {-1} × V becomes the identity matrix I, which is simple. $
3-1-8. Singular value decomposition
- Do something similar to eigenvalue decomposition other than a square matrix.
M\overrightarrow {v}=\sigma \overrightarrow {u}\\
M^{T}\overrightarrow {u}=\sigma \overrightarrow {v}\\
T is the ** transpose ** of M. If there is such a special unit vector, it can be decomposed into singular values.
M=USV^{-1}
U and V are orthogonal matrices (unitary matrices if they have complex numbers as elements)
3-1-9. How to find the singular value
MV = US ⇔ M^{T}U = VS^{T}\\
(Does this have this relationship?)\\
M = USV^{-1} M^{T} = VS^{T}U^{-1}\\
These products are
MM^{T} = USV^{-1}VS^{T}U^{-1} = USS^{T}U^{-1}\\
In a concrete example
M =\begin{pmatrix}
1 & 2 & 3 \\
3 & 2 & 1
\end{pmatrix}\
=\begin{pmatrix}
\dfrac {1}{\sqrt {2}} & -\dfrac {1}{\sqrt {2}} \\
\dfrac {1}{\sqrt {2}} & \dfrac {1}{\sqrt {2}}
\end{pmatrix}\begin{pmatrix}
2\sqrt {6} & 0 & 0 \\
0 & 2 & 0
\end{pmatrix}\begin{pmatrix}
\dfrac {1}{\sqrt {3}} & \dfrac {1}{\sqrt {3}}\dfrac {1}{\sqrt {3}} \\
\dfrac {1}{\sqrt {2}} & 0-\dfrac {1}{\sqrt {2}} \\
\dfrac {1}{\sqrt {6}} & -\dfrac {2}{\sqrt {6}}\dfrac {1}{\sqrt {6}}
\end{pmatrix}
3-1-10. How to use singular value decomposition
In machine learning, etc., from a matrix of image data decomposed by singular value If you remove the small part of the component, ** the image becomes blurry, but the amount of data can be reduced. ** ** ⇒ Blurred image by shifting the focus of the lens (the amount of data is reduced)
What is a deep learning course that can be crushed in the field in 3 months
Recommended Posts