[PYTHON] I tried to solve the E qualification problem collection [Chapter 1, 5th question]

This is the first post.

I am studying for E qualifications sponsored by JDLA (Japan Deep Learning Association). As a countermeasure, we use "Thorough Strategy Deep Learning E Qualification Engineer Problem Collection". However, when solving the problem, typographical errors are sometimes found. Since this is the only E-qualification countermeasure book, I would like to write one question per article for those who are studying from now on. Please point out any mistakes.

Chapter 1 5th question

First, let's look at the question sentences and answers in the text. Next, I will post the result I solved and the answer of the poster.

Text question sentence

Select one option that applies to (a) to (d) of the following formula. However, the same option may apply in more than one place.

queue

A=
\begin{pmatrix}
1 & 0 & 0\\
0 & 1 & 2
\end{pmatrix}

Is decomposed into singular values ​​and expressed in the form of $ A ＝ U \ Sigma V ^ T $, as follows.

\Sigma =
\begin{pmatrix}
(A) & 0 & 0\\
0 & (I) & 0
\end{pmatrix}
\\
U=
\begin{pmatrix}
0 & 1\\
(C) & 0
\end{pmatrix}
\\
V=
\begin{pmatrix}
0 & 1 & 0\\
\frac{1}{\sqrt{5}} & 0 & -\frac{2}{\sqrt{5}}\\
\frac{2}{\sqrt{5}} & 0 & (D)
\end{pmatrix}

A. -1 B. 0 C. 1 D. 2 E. -\sqrt{5} F. \sqrt{5} G. -\frac{1}{\sqrt{5}} H. \frac{1}{\sqrt{5}}

Text answer

** (A) C, (B) F, (C) C, (D) H **

First, $ A \ for the singular value of the m × n rectangular matrix $ A $, that is, the $ (i, i) $ component (i = 1, ..., min {m, n}) of $ \ Sigma $. The positive square roots of the ^ TA $ eigenvalues ​​are arranged in descending order (largest order).

A^T A=
\begin{pmatrix}
1 & 0\\
0 & 5
\end{pmatrix}

So its eigenvalue is

det(\lambda I - A ^T A) = det
\begin{pmatrix}
\lambda -1 & 0\\
0 & \lambda -5
\end{pmatrix}
= 0

Solving, $ \ lambda = 1, 5 $. The singular value of $ A $ is $ \ sigma = 1, \ sqrt {5} $, so if you arrange them in descending order, (A) is 1 and (B) is $ \ sqrt {5} $ (** A = C, i = F **). Here, the first column of $ U $ is the eigenvector $ u = (() u_x, u_y) ^ T $ corresponding to the eigenvalue 5 of $ A ^ T A $, which has a magnitude of 1. Therefore,

\begin{pmatrix}
1 & 0\\
0 & 5
\end{pmatrix}
\begin{pmatrix}
u_x\\
u_y
\end{pmatrix}
=5
\begin{pmatrix}
u_x\\
u_y
\end{pmatrix}

To get the solution that $ u_x = 0 $ and $ u_y $ are arbitrary real numbers. Here we can see that $ u_y = \ pm1 $ must be in order for $ u $ to be 1 in size. Also, each column of $ V $ is an eigenvector of $ A ^ T A $, but these must satisfy ** orthonormality **. So put $ v_1 = (0, 1/\ sqrt {5}, 2/\ sqrt {5}) ^ T $, $ v_3 = (0, -2/\ sqrt {5}, v_z) ^ T $ For example, $ v_1 ^ T \ cdot v_3 = 0 $ due to their orthogonality. Therefore,

0 \cdot 0 + \frac{1}{\sqrt{5}} \cdot \left( -\frac{2}{\sqrt{5}} \right) + \frac{2}{\sqrt{5}} v_z = 0

Then, (d) $ v_z = 1/\ sqrt {5} $ is obtained (** d = H ). Finally, you can confirm by substitution whether the value that applies to (c) is 1 or -1, and identify it as 1. ( C = C **).

I tried to solve it myself

When I tried to solve this problem myself, the text answer and the choices were different. There is a good chance that your calculation is wrong, so substitute the options in the text answer for $ \ Sigma $, $ U $, and $ V $ to calculate $ U \ Sigma V ^ T $. When I tried it, it became as follows.

\begin{eqnarray}
U \Sigma V^T
&=&
\begin{pmatrix}
0 & 1\\
1 & 0
\end{pmatrix}
\begin{pmatrix}
1 & 0 & 0\\
0 & \sqrt{5} & 0
\end{pmatrix}
\begin{pmatrix}
0 & \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}}\\
1 & 0 & 0\\
0 & -\frac{2}{\sqrt{5}} & \frac{1}{\sqrt{5}}
\end{pmatrix}\\
&=&
\begin{pmatrix}
0 & \sqrt{5} & 0\\
1 & 0 & 0
\end{pmatrix}
\begin{pmatrix}
0 & \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}}\\
1 & 0 & 0\\
0 & -\frac{2}{\sqrt{5}} & \frac{1}{\sqrt{5}}
\end{pmatrix}\\
&=&
\begin{pmatrix}
\sqrt{5} & 0 & 0\\
0 & -\frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}}
\end{pmatrix}
\end{eqnarray}

It has a different composition than $ A $. I am looking for a singular value of $ A $ in the answer, but it is probably because "(A) is 1 and (B) is $ \ sqrt {5} $".

Let's calculate $ U \ Sigma V ^ T $ again in the presence of unknowns. Let (a) be w, (b) be x, (c) be y, and (d) be z. Then

\begin{eqnarray}
U \Sigma V^T
&=&
\begin{pmatrix}
0 & 1\\
y & 0
\end{pmatrix}
\begin{pmatrix}
w & 0 & 0\\
0 & x & 0
\end{pmatrix}
\begin{pmatrix}
0 & \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}}\\
1 & 0 & 0\\
0 & -\frac{2}{\sqrt{5}} & z
\end{pmatrix}\\
&=&
\begin{pmatrix}
0 & x & 0\\
wy & 0 & 0
\end{pmatrix}
\begin{pmatrix}
0 & \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}}\\
1 & 0 & 0\\
0 & -\frac{2}{\sqrt{5}} & z
\end{pmatrix}\\
&=&
\begin{pmatrix}
x & 0 & 0\\
0 & \frac{wy}{\sqrt{5}} & \frac{2wy}{\sqrt{5}}
\end{pmatrix}
\end{eqnarray}

If you compare this with the $ (1,1) $ component of $ A $, you can see that $ x = 1 $. Since the singular values ​​of $ A $ were 1 and $ \ sqrt {5} $, we know that $ w = \ sqrt {5} $. By comparing the $ (2,2) $ components, we can see that $ y = 1 $. Due to the nature of singular value decomposition, $ V $ is an orthonormal matrix, so if we take the inner product of the vector $ v_1 $ in the first column and the vector $ v_3 $ in the third column,

v_1 \cdot v_3
=
\begin{pmatrix}
0\\
\frac{1}{\sqrt{5}}\\
\frac{2}{\sqrt{5}}
\end{pmatrix}
\cdot
\begin{pmatrix}
0\\
-\frac{2}{\sqrt{5}}\\
z
\end{pmatrix}
=
0 \cdot 0 - \frac{1}{\sqrt{5}}\ \cdot \frac{2}{\sqrt{5}} + \frac{2}{\sqrt{5}} z
=
-\frac{2}{5} + \frac{2}{\sqrt{5}} z
=
0

Solving this, we find $ z = 1/\ sqrt {5} $. From the above, the answer of the poster is as follows.

Poster's answer

** (A) F, (B) C, (C) C, (D) H **

As an aside, maybe $ u = (() u_x, u_y) ^ T $ is just a typo, and I think it's $ u = (u_x, u_y) ^ T $.