[PYTHON] Beginning with PyTorch

PyTorch

PyTorch: A deep learning framework that can calculate neural networks.

I summarized the procedure up to installation to build the environment of PyTorch. (I may write a simple usage someday)

Install PyTorch

pyenv + anaconda3-5.3.1 on CentOS Linux release 7.7.1908 (Core) I always install pyenv on Linux and use anaconda 3-5.3.1. Looking at the original page, it seems that you can also enter using the conda command, for example

#Stable on the original site(1.5), Linux, Conda, Python, None (cuda options)Commands displayed with
#If you want to install the latest version, you should regenerate the command on the original site
conda install pytorch torchvision cpuonly -c pytorch

Then PyTorch enters. However, if PyTorch is included in this procedure, at least in my environment (CentOS Linux release 7.7.1908 (Core)), it will destroy the environment of ipython and jupyter in anaconda. </ font> This is the end of the story. Therefore, I searched for another way.

pyenv + 3.8.2 on CentOS Linux release 7.7.1908 (Core) Looking at the page of the head family, it says that you can also enter with pip. In that case, I decided to put in a simple python and add it with pip from there. If you're not interested in the process and just want to install it, go to the final summary of this subsection.

Put pyenv (see the infinite number of articles you can find in'pyenv installation'for how to put it) and put it in your PATH (written in the infinite number of articles). When I checked with pyenv install --list, 3.8.2 was the latest in my pyenv, so I decided to select this.

pyenv install 3.8.2

When I try to install with **, I cannot install. ** If you check for errors and warnings (it's too lazy to rebuild later, so warnings are supported as long as COSPA allows), it seems that zlib, ssl, bzip2, realine, SQlite are enough. There seems to be no. Before this, I should have included gcc,g ++, gfortran, but there may have been other apps / libraries already included. (Anaconda doesn't refer to the system libraries, so I wonder if there was any problem with the installation so far. I'm not sure.) When I try to install these libraries and try yum install zlib etc., it says that it already exists. .. So all you need is zlib-dev. This is mostly true for other libraries as well. So

yum install zlib-*    #zlib-*With zlib-Insert dev etc.
yum install openssl-*
yum install bzip2-*
yum install readline-*
yum install sqlite-*

After that, I put in the above python installation command and it was built successfully. I've only used anaconda, so I thought I was just decompressing a compressed binary called pyenv, but I'm building it. Also, the fact that * -dev is needed may mean that the static library is linked at build time.

Well, that's why

#Stable on the original site(1.5), Linux, Pip, Python, None (cuda options)Commands displayed with
#If you want to install the latest version, you should regenerate the command on the original site
pip install torch==1.5.0+cpu torchvision==0.6.0+cpu -f https://download.pytorch.org/whl/torch_stable.html

When I thought that it would be fine and PyTorch would come in, it would be moss. Apparently ctypes can't be read well. As a test, you can throw ʻimport ctypes from python3withModuleNotFoundError: No module named'_ctypes'error. Sincectypesis a fairly basic library, it's possible that it wasn't included in the 3.8.2 build, so something is fundamentally wrong. When I checked various error messages, it seems thatlibffiis missing. And history repeats itself.libffi itself is already installed and all you need is libffi-dev`. so,

yum install libffi-*

do. Actually, this alone doesn't solve the problem, ** this library must be linked when building python ** (quickly). So, after deleting it with pyenv uninstall 3.8.2, you will be able to import ctypes properly with pyenv install 3.8.2. Up to this point, the installation command with pip of PyTorch written above will pass. After that, if you put jupyter, matplotlib, scipy in pip, you can use most of the functions. numpy is probably included as a PyTorch installation dependency.

Summary

I wrote about the path of PyTorch installation that has been discovered and solved. But if you know the answer from the beginning

yum install zlib-* openssl-* bzip2-* readline-* sqlite-* libffi-*
pip install torch==1.5.0+cpu torchvision==0.6.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
pip install jupyter
pip install matplotlib
pip install scipy

So, you should have a minimum development environment with PyTorch. (For gcc g ++, if you haven't installed it yet, you may have to yum install as well.)

If you don't like to get in touch with Anaconda, you can set it to pyenv local 3.8.2 in the directory that uses PyTorch. If you move to another directory, the environment specified as global will be created.

pyenv + 3.8.2 on Ubuntu 19.10 There were various misunderstandings, and if I did pyenv 8.3.2 with only sudo apt install libffi-dev installed,

WARNING: The Python bz2 extenWARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?sion was not compiled. Missing the bzip2 lib?

I got an error like that. In case of Ubuntu, the above yum corresponds to

sudo apt install libbz2-dev
sudo apt install libreadline-dev 
sudo apt install libsqlite3-dev

become. This will install python 3.8.2, so all you have to do is add PyTorch, jupyter, matplotlib, and scipy with pip. Referenced site: [WARNING occurs when trying to install Python 3.7.0 with pyenv --Programming memo blog of automation kitchen │ CODE-LIFE](https://code-life.hatenablog.com/entry/pyenv- install-3.7.0)

Torch tensor

--Get shape with torch_array.shape --Get length with torch_array.size

Creating a model

An example of a model with two hidden layers. In nn.Linear, the activation function is ReLU.

import torch
import torch.nn as nn
import torch.nn.functional as F
class Model_2L(nn.Module):
    def __init__(self, input_size, output_size, ndim1 = 10, ndim2=10):
        super(Model_2L,self).__init__()
        self.n1 = ndim1
        self.n2 = ndim1
        self.fc1 = nn.Linear(input_size, self.n1)
        self.fc2 = nn.Linear(self.n1, self.n2)
        self.fc3 = nn.Linear(self.n2, output_size)

    def forward(self, x):
        x = self.fc1(x)
        x = self.fc2(F.relu(x))
        x = self.fc3(F.relu(x))
        return x
model = Model_3L(input_size, output_size)

trouble shooting

RuntimeError: size mismatch, m1: [1 x 15], m2: [1 x 1] at /pytorch/aten/src/TH/generic/THTensorMath.cpp:41

The shape of the argument torch tensor is probably strange. In my case, I made it with numpy ndarray and then changed it to torch tensor with torch.from_numpy, and the declaration at the time of numpy ndarray was strange. Specifically, what should be made with (Ndim, 1) was made with (Ndim,). This transformation can be done with x_array = x_array.reshape (Ndim, -1).

RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #2 'mat1' in call to _th_addmm Probably, the type of the input tensor is not 32bit real number, it is 64bit real number, but the problem is that the read function does not support 64bit real number.

According to 2nd PyTorch Tensor & Data Type Cheat Sheet: Introduction to PyTorch-@IT

There are many types, but basically only 32bit torch.float or torch.int is used.

It seems that. I think many numpys use float64 by default, so you need to explicitly add dtype = float32 in the declaration part or convert it to a 32bit real number with x_array = np.float32 (x_array). is there.

Recommended Posts

Beginning with PyTorch
Play with PyTorch
Cross-validation with PyTorch
Use RTX 3090 with PyTorch
Try an autoencoder with Pytorch
Try implementing XOR with PyTorch
Implement PyTorch + GPU with Docker
Prediction of Nikkei 225 with Pytorch 2
Machine learning Minesweeper with PyTorch
AWS Lambda with PyTorch [Lambda import]
Prediction of Nikkei 225 with Pytorch
Perform Stratified Split with PyTorch
Beginning with Python machine learning
I made Word2Vec with Pytorch
[PyTorch Tutorial ⑤] Learning PyTorch with Examples (Part 2)
Learn with PyTorch Graph Convolutional Networks
I implemented Attention Seq2Seq with PyTorch
Prediction of Nikkei 225 with Pytorch ~ Intermission ~
How to Data Augmentation with PyTorch
[PyTorch Tutorial ⑤] Learning PyTorch with Examples (Part 1)
pytorch @ python3.8 environment construction with pipenv
Achieve pytorch reflection padding with Tensorflow
Sine wave prediction (regression) with Pytorch
Install pytorch
Multi-class, multi-label classification of images with pytorch
I implemented Shake-Shake Regularization (ShakeNet) with PyTorch
PyTorch Links
Make a drawing quiz with kivy + PyTorch
[Python] Introduction to CNN with Pytorch MNIST
Document classification with toch text from PyTorch
[Introduction to Pytorch] I played with sinGAN ♬
I tried batch normalization with PyTorch (+ note)
I tried implementing DeepPose with PyTorch PartⅡ
I tried to implement CVAE with PyTorch
Machine learning with Pytorch on Google Colab
Install PyTorch
[PyTorch] Handle image pairs with Dataset & DataLorder
I tried to detect Mario with pytorch + yolov3
I tried to implement reading Dataset with PyTorch
I rewrote Chainer's MNIST code with PyTorch + Ignite
Story of trying to use tensorboard with pytorch
Display the image after Data Augmentation with Pytorch