I bought this book because I wanted to learn about PyTorch.
After comparing some of them, I chose the one for beginners, but ... I don't know! !! !! It became .
_ ** I haven't written the output, so I don't know what is output and what kind of viewpoint to check! !! ** _
So, I've put together a ** supplement ** that says this is probably what I'm trying to say. The execution environment is Google Colaboratory.
As a preliminary preparation, install PyTorch.
IN
import torch
PyTorch uses the GPU. Let's check if the GPU is available.
IN
#Check if GPU is available
print(torch.cuda.is_available())
OUT
True
According to the book, Tensor is
--A data structure for handling multidimensional arrays --Has almost the same API as Numpy's ndarray --Supports calculations on GPU
That's right.
In short, it's a PyTorch multidimensional array.
In the book, there were explanations and sample code of mess and data types, but I just defined it ** I have not confirmed the contents at all **.
So, ** I will check the data **.
IN
#Create by passing a nested list
t = torch.tensor([[1, 2], [3, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t
OUT
dtype: torch.float32
device: cpu
tensor([[1., 2.],
[3., 4.]])
dtype is the data type and device is whether the Tensor is on the CPU or GPU.
Let's make a Tensor on the GPU.
IN
t = torch.tensor([[1, 2], [2, 4.]], device="cuda:0")
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t
OUT
dtype: torch.float32
device: cuda:0
tensor([[1., 2.],
[2., 4.]], device='cuda:0')
It seems that you can also create a Tensor on the GPU with torch.cuda.FloatTensor.
IN
t = torch.cuda.FloatTensor([[1, 2], [2, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t
dtype: torch.float32
device: cuda:0
tensor([[1., 2.],
[2., 4.]], device='cuda:0')
By the way, cuda is according to Wikipedia
CUDA (Compute Unified Device Architecture) is a general-purpose parallel computing platform (parallel computing architecture) and programming model for GPUs developed and provided by NVIDIA.
That's right.
Now let's change the data type.
IN
#Specify 64-bit data type
t = torch.tensor([[1, 2], [2, 4.]], dtype=torch.float64)
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t
OUT
dtype: torch.float64
device: cpu
tensor([[1., 2.],
[2., 4.]], dtype=torch.float64)
A 64-bit signed integer can also be defined with torch.LongTensor.
IN
t = torch.LongTensor([[1, 2], [2, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t
OUT
dtype: torch.int64
device: cpu
tensor([[1, 2],
[2, 4]])
The data types are summarized in this document. https://pytorch.org/docs/stable/tensors.html
Try transferring the Tensor created on the CPU to the GPU.
IN
t = torch.zeros(100, 10).to("cuda:0")
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t[:2] #Slices are also available in Tensor
OUT
dtype: torch.float32
device: cuda:0
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]], device='cuda:0')
If you want to see the shape of the Tensor, use the size function instead of the shape function.
This is different from NumPy.
IN
#shape is obtained by size
# t.shape() #Compile error
t.size()
OUT
torch.Size([100, 10])
Use the numpy function to convert a tensor to a Numpy ndarray.
IN
t = torch.tensor([[1, 2], [3, 4.]])
x = t.numpy()
print("type: {}".format(type(x)))
x
OUT
type: <class 'numpy.ndarray'>
array([[1., 2.],
[3., 4.]], dtype=float32)
The Tensor on the GPU needs to be converted to a CPU.
IN
t = torch.cuda.FloatTensor([[1, 2], [3, 4.]])
x = t.to("cpu").numpy()
print("type: {}".format(type(x)))
x
OUT
type: <class 'numpy.ndarray'>
array([[1., 2.],
[3., 4.]], dtype=float32)
Let's try a confusing way to convert a tensor.
Reshape in Numpy is executed by view function.
IN
# 2 *2 to 4*Change to 1
#view is similar to reshape of ndarray
x1 = torch.tensor([[1, 2], [3, 4.]])
x1.view(4, 1)
OUT
tensor([[1.],
[2.],
[3.],
[4.]])
There are several ways to write transpose.
IN
x2 = torch.tensor([[10, 20, 30], [40, 50, 60.]])
x2.T
OUT
tensor([[10., 40.],
[20., 50.],
[30., 60.]])
IN
x2.t()
OUT
tensor([[10., 40.],
[20., 50.],
[30., 60.]])
Transpose can also be transposed, but in addition to transposing, it can also be used to sort the image data format from HWC (vertical, horizontal, color) to CHW (color, horizontal, vertical).
IN
hwc_img_data = torch.rand(1, 5, 4, 3)
hwc_img_data
OUT
tensor([[[[0.9248, 0.7545, 0.5603],
[0.5339, 0.6627, 0.7652],
[0.7082, 0.5146, 0.9273],
[0.8437, 0.7064, 0.1053]],
[[0.2080, 0.8018, 0.6833],
[0.5892, 0.9264, 0.9315],
[0.0872, 0.1898, 0.5745],
[0.2192, 0.1187, 0.7537]],
[[0.9680, 0.9239, 0.8698],
[0.2339, 0.9918, 0.3446],
[0.6669, 0.4148, 0.2037],
[0.1055, 0.0353, 0.3679]],
[[0.7079, 0.4069, 0.1181],
[0.1983, 0.0452, 0.5788],
[0.6378, 0.7050, 0.1148],
[0.3960, 0.1924, 0.2714]],
[[0.3127, 0.1320, 0.7232],
[0.3484, 0.7617, 0.4725],
[0.4863, 0.9178, 0.3092],
[0.6279, 0.4259, 0.3828]]]])
I somehow understood that I would change from 1 * 5 * 4 * 3 to 1 * 4 * 5 * 3 (5 and 4 would be interchanged).
IN
# 1 * 5 * 4 *3 to 1* 4 * 5 *Change to 3
hwc_img_data.transpose(1, 2)
OUT
tensor([[[[0.9248, 0.7545, 0.5603],
[0.2080, 0.8018, 0.6833],
[0.9680, 0.9239, 0.8698],
[0.7079, 0.4069, 0.1181],
[0.3127, 0.1320, 0.7232]],
[[0.5339, 0.6627, 0.7652],
[0.5892, 0.9264, 0.9315],
[0.2339, 0.9918, 0.3446],
[0.1983, 0.0452, 0.5788],
[0.3484, 0.7617, 0.4725]],
[[0.7082, 0.5146, 0.9273],
[0.0872, 0.1898, 0.5745],
[0.6669, 0.4148, 0.2037],
[0.6378, 0.7050, 0.1148],
[0.4863, 0.9178, 0.3092]],
[[0.8437, 0.7064, 0.1053],
[0.2192, 0.1187, 0.7537],
[0.1055, 0.0353, 0.3679],
[0.3960, 0.1924, 0.2714],
[0.6279, 0.4259, 0.3828]]]])
IN
#1 more* 4 * 5 *3 to 1* 3 * 4 *Change to 5
#Now you can convert from hwc to cwh
hwc_img_data.transpose(1, 2).transpose(1, 3)
OUT
tensor([[[[0.9248, 0.5339, 0.7082, 0.8437],
[0.2080, 0.5892, 0.0872, 0.2192],
[0.9680, 0.2339, 0.6669, 0.1055],
[0.7079, 0.1983, 0.6378, 0.3960],
[0.3127, 0.3484, 0.4863, 0.6279]],
[[0.7545, 0.6627, 0.5146, 0.7064],
[0.8018, 0.9264, 0.1898, 0.1187],
[0.9239, 0.9918, 0.4148, 0.0353],
[0.4069, 0.0452, 0.7050, 0.1924],
[0.1320, 0.7617, 0.9178, 0.4259]],
[[0.5603, 0.7652, 0.9273, 0.1053],
[0.6833, 0.9315, 0.5745, 0.7537],
[0.8698, 0.3446, 0.2037, 0.3679],
[0.1181, 0.5788, 0.1148, 0.2714],
[0.7232, 0.4725, 0.3092, 0.3828]]]])
If the requires_grad attribute is set to True, automatic differentiation will be performed. The book doesn't describe in detail where and what will appear, so I'll add it.
IN
x = torch.randn(100, 3)
a = torch.tensor([1, 2, 3.], requires_grad=True)
y = torch.mv(x, a)
print(y.grad_fn) #Make sure that the calculation graph for calculating the gradient is stored
o = y.sum()
#Get the gradient
#Variables that perform backwards must be scalar
o.backward()
#Automatic differentiation is performed, a.Gradient is acquired in grad
print(a.grad)
a.grad != x.sum(0)
OUT
<MvBackward object at 0x7f9c7998e3c8>
tensor([ 2.5016, 6.3925, -6.3674])
tensor([False, False, False])
Recommended Posts