[PYTHON] Matrix considered by function
Last time, I saw that covectors can be used to combine multiple functions that return scalars. This time, the function that returns a vector is represented by a matrix. [Algebraic Dual Space](https://en.wikipedia.org/wiki/%E5%8F%8C%E5%AF%BE%E3%83%99%E3%82%AF%E3%83%88%E3 The purpose is to convey the image of% 83% AB% E7% A9% BA% E9% 96% 93). Use Python as the programming language for comparison and attach the NumPy results to the calculations.
This is a series of articles.
- Inner product considered by estimation
- Covector to think with functions
- Matrix considered by function ← This article
- Duality in function
- Perceptron Thinking in Covector
Covector function
I will repost the covector that returns the value specified from the three arguments introduced last time.
- Hereafter, ʻimport` will be omitted.
>>> from numpy import *
>>> GetX = [1, 0, 0]
>>> GetY = [0, 1, 0]
>>> GetZ = [0, 0, 1]
>>> dot(GetX, [1, 2, 3])
1
>>> dot(GetY, [1, 2, 3])
2
>>> dot(GetZ, [1, 2, 3])
3
\begin{align}
GetX
&\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}1 & 0 & 0\end{matrix}\right)
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=1 \\
GetY
&\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}0 & 1 & 0\end{matrix}\right)
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=2 \\
GetZ
&\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}0 & 0 & 1\end{matrix}\right)
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=3
\end{align}
Many to one
Last time I saw that you can pass the same arguments to multiple functions. I will try it with the previous function.
>>> dot([GetX,GetY,GetZ],[1,2,3])
array([1, 2, 3])
\begin{align}
\left(\begin{matrix}GetX \\ GetY \\ GetZ\end{matrix}\right)
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
&=\left(\begin{array}{r}
GetX \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) \\
GetY \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) \\
GetZ \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
\end{array}\right) \\
&=\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
\end{align}
If you consider the entire return value as a vector, you will notice that it is returning the arguments as they are. It is a natural result to note that the matrix composed of covectors becomes the identity matrix.
\left(\begin{matrix}GetX \\ GetY \\ GetZ\end{matrix}\right)
=\left(\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{matrix}\right)
Matrix function
If you think of the previous example as one result of one function, you can think of the matrix composed of covectors as one function.
>>> Id=[GetX,GetY,GetZ]
>>> dot(Id,[1,2,3])
array([1, 2, 3])
Id
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
You can also create a function to cycle by shifting the arrangement.
>>> Rotate=[GetY,GetZ,GetX]
>>> dot(Rotate,[1,2,3])
array([2, 3, 1])
Rotate
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}2 \\ 3 \\ 1\end{matrix}\right)
In this way, you can think of a matrix as a function and interpret it by dividing it into covectors as needed.
Bondability
It is also possible to shift it twice in a row.
>>> dot(Rotate,dot(Rotate,[1,2,3]))
array([3, 1, 2])
Rotate\left(Rotate
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)\right)
=\left(\begin{matrix}3 \\ 1 \\ 2\end{matrix}\right)
It is possible to calculate the product of Rotates first, just as the result is the same even if the combination to be calculated first is changed such as $ a (bc) = (ab) c $ by multiplication.
>>> dot(dot(Rotate,Rotate),[1,2,3])
array([3, 1, 2])
\left\{(Rotate)(Rotate)\right\}
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}3 \\ 1 \\ 2\end{matrix}\right)
The fact that the result does not change even if the calculation order is changed in this way is called ** connectivity **. The method of calculating the product of matrices is designed to ensure associativity.
Function synthesis
It is possible to consider the product of functions by a matrix as ** function composition ** and assign it to a new function.
>>> Rotate2=dot(Rotate,Rotate)
>>> dot(Rotate2,[1,2,3])
array([3, 1, 2])
Rotate2
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}3 \\ 1 \\ 2\end{matrix}\right)
If you Rotate three times, it will return to the original, but you can see that the result of function synthesis is an identity matrix.
>>> Rotate3=dot(Rotate2,Rotate)
>>> dot(Rotate3,[1,2,3])
array([1, 2, 3])
>>> Rotate3
array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])
(Rotate2)(Rotate)
=\left(\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{matrix}\right)
Number of inputs / outputs
If you think of a matrix as a function, you can interpret that the size of the matrix represents the number of elements in the argument (number of inputs) and the number of elements in the return value (number of outputs).
Input → Function → Output
(X, X) -> F -> (X, X, X)
(X, X, X) -> G -> X
\begin{align}
F&:\quad\scriptsize{Number of outputs 3}\normalsize{\Biggr\{
\overbrace{\left(\begin{matrix}f_{11} & f_{12} \\ f_{21} & f_{22} \\ f_{31} & f_{32}\end{matrix}\right)}^{Number of inputs 2}} \\
G&:\quad\scriptsize{Number of outputs 1}\normalsize{\{
\overbrace{\left(\begin{matrix}g_1 & g_2 & g_3\end{matrix}\right)}^{Number of inputs 3}}
\end{align}
If you want to calculate $ F $ and $ G $ in succession, the number of outputs of $ F $ calculated first must match the number of inputs of subsequent $ G $.
F→
\underbrace{\left(\begin{matrix}t_1 \\ t_2 \\ t_3\end{matrix}\right)}_{Value in the middle}
→G
Check the component display to see the situation where $ F $ and $ G $ are combined into $ GF $.
- Note that the order of function composition $ GF $ is the reverse of the calculation order $ F $ → $ G $.
\underbrace{GF}_{Synthetic}
=\overbrace{\left(\begin{matrix}g_1 & g_2 & g_3\end{matrix}\right)}^{Number of inputs 3}
\quad\scriptsize{Number of outputs 3}\normalsize{\Biggr\{
\left(\begin{matrix}f_{11} & f_{12} \\ f_{21} & f_{22} \\ f_{31} & f_{32}\end{matrix}\right)}
Calculation flow
The flow of calculation including input and output is shown. The red number represents the number of elements.
Schematize by focusing on the value.
\underbrace{y}_{output}
\xleftarrow{G}
\underbrace{\left(\begin{matrix}t_1 \\ t_2 \\ t_3\end{matrix}\right)}_{Value in the middle}
\xleftarrow{F}
\underbrace{\left(\begin{matrix}x_1 \\ x_2\end{matrix}\right)}_{input}
Draw a figure in the style of Perceptron. The calculation flow is from right to left according to the matrix notation.
Notice the line connecting $ t $ and $ x $. Taking $ f_ {21} $ as an example, we can see that by interpreting the subscript as $ 2 ← 1 $, it corresponds to the subscript of the node $ t_2 ← x_1 $ connected by the line. The schematic is as follows.
t_2 \xleftarrow{f_{21}} x_1
Make sure that the other lines have the same pattern.
Tensor flow
- This section is a digression. You don't have to know.
If you use the node subscript as a variable and write it as $ x_i, t_j $, the line connecting them will be $ f_ {ji} $.
t_j \xleftarrow{f_{ji}} x_i
If you include the last step, the flow will be as follows.
y \xleftarrow{g_j} t_j \xleftarrow{f_{ji}} x_i
** Tensor algebra ** expresses this relationship with the following formula.
t_j=f_{ji}x_i \\
y=g_jt_j=g_jf_{ji}x_i
The representation of tensor algebra is a matrix with subscript relationships added.
\boldsymbol{t}=\boldsymbol{Fx} \\
y=\boldsymbol{Gt}=\boldsymbol{GFx}
- In Tensor algebra, [Einstein notation](https://ja.wikipedia.org/wiki/%E3%82%A2%E3%82%A4%E3%83%B3%E3%82%B7%E3 % 83% A5% E3% 82% BF% E3% 82% A4% E3% 83% B3% E3% 81% AE% E7% B8% AE% E7% B4% 84% E8% A8% 98% E6% B3 By using% 95), it looks the same except for matrix expressions and subscripts. Details are omitted because it is beyond the scope of this time.
If you express the node value in tensor algebra, you can see the flow of the tensor.
\underbrace{g_jf_{ji}x_i}_{output}
\xleftarrow{g_j}
\underbrace{f_{ji}x_i}_{Value in the middle}
\xleftarrow{f_{ji}}
\underbrace{x_i}_{input}
