[PYTHON] Understand the tensor product (numpy.tensordot)

I didn't understand numpy's tensor dot catastrophically, but I think I could understand it by researching various things, so I'll make a note of it.

This subject

>>> import numpy as np
>>> a = np.arange(12).reshape(2,3,2)
>>> b = np.arange(48).reshape(3,2,8)
>>> c = np.tensordot(a,b, axes=([1,0], [0,1]))

>>> a
array([[[ 0,  1],
        [ 2,  3],
        [ 4,  5]],

       [[ 6,  7],
        [ 8,  9],
        [10, 11]]])

>>> b
array([[[ 0,  1,  2,  3,  4,  5,  6,  7],
        [ 8,  9, 10, 11, 12, 13, 14, 15]],

       [[16, 17, 18, 19, 20, 21, 22, 23],
        [24, 25, 26, 27, 28, 29, 30, 31]],

       [[32, 33, 34, 35, 36, 37, 38, 39],
        [40, 41, 42, 43, 44, 45, 46, 47]]])

>>> c
array([[ 800,  830,  860,  890,  920,  950,  980, 1010],
       [ 920,  956,  992, 1028, 1064, 1100, 1136, 1172]])

What we are doing with np.tensordot 1: Generate an intermediate matrix from matrix a

The image is that the elements are picked up in the order of the second dimension and the first dimension of the matrix a, and an intermediate matrix for calculation is created. The shape of the intermediate matrix is the number of the second dimension x the number of the first dimension.

In this case, it is an image that an intermediate matrix with a shape of 3 (second dimension number) x 2 (first dimension number) is created.

In any case, pick up the top left element first.

Which element to pick up next? Since it is picked up in the order of the second dimension and the first dimension, First, pick up in the order of the second dimension.

It will be like this when picked up in the order of the second dimension.

Furthermore, which element to pick up next is I picked it up in the order of the second dimension, so Next, jump to the "lower" island in the order of the first dimension.

As before, pick up the elements in the second dimensional order on the "lower" island.

You now have a 3x2 matrix.

Do the same for the "right" side.

You now have a second 3x2 matrix.

As a result, the following structure is created.

What we are doing with np.tensordot 2: Generate an intermediate matrix from matrix b

The image is that the elements are picked up in the order of the first dimension and the second dimension of the matrix b, and an intermediate matrix for calculation is created.

The shape of the intermediate matrix is the number of the first dimension x the number of the second dimension.

In this case, it is an image that an intermediate matrix with a shape of 3 (first dimension number) x 2 (second dimension number) is created.

In any case, pick up the top left element first.

Which element to pick up next? Since it is picked up in the order of the first dimension and the second dimension, Next, you have to jump to the "bottom" island.

It will be like this when picked up in the order of the first dimension.

Furthermore, which element to pick up next is I picked it up in the order of the first dimension, so Next, we will return to the "top" island in the order of the second dimension.

While restarting from ④, pick up the elements in the order of the first dimension as before.

You now have a 3x2 matrix.

Do the same for the rest.

You should have eight 3x2 matrices.

As a result, the following structure is created.

What we are doing with np.tensordot 3: Take the "product" of the two intermediate matrices obtained from the matrix a and the matrix b.

It has been made compact into a 2D x 8D matrix.

What are you doing conceptually

By calculating for these dimensions, we can reduce these dimensions while preserving information on these dimensions.

It is compacted to the shape of these remaining dimensions (that is, 2 dimensions x 8 dimensions).

In this sense, it seems that such an operation is called contraction. I interpret it as "reduction + summary".

The tensor product seems to be like this.