[PYTHON] Steps to build PyTorch 1.5 for CUDA 10.2 on Windows

Here are the steps to build PyTorch 1.5 for CUDA 10.2 on Windows 10. (As of March 1, 2020) It also includes the torchvision build instructions that are commonly used with PyTorch. Detailed explanation is omitted here, so if you need a little more detailed explanation, please also refer to Blog --dev.infohub.cc. I hope you can.

Build environment

• hardware
  • CPU: AMD Threadripper 3960X
  • GPU: nVIDIA TITAN RTX --Software
  • Windows 10 Pro Version 1909 (x64)
  • Microsoft Visual Studio Community 2019 – Ver 16.4.5 --Git for Windows (anything is OK if you can use the git command) --Python 3.8.2 (for Python.org, not Anaconda)
  • CUDA 10.2
  • cuDNN 7.6.5 --Intel MKL 2019.0 (AMD CPU but forcibly use MKL)

Settings such as environment variables

--MKL_DEBUG_CPU_TYPE = 5 (Settings for using MKL on AMD CPUs) --The PATH to other CUDA etc. is in place

Build PyTorch, torchvision

Open the x64 Native Tools Command Prompt for VS 2019 in Visual Studio 2019 and follow the steps below. (Explanation is posted in comment format)

#Virtual environment for build(build_pt)Create
python -m venv g:\work\build_pt
g:\work\build_pt\Scripts\activate.bat
cd /d g:\work\build_pt

#Updates such as packages
python -m pip install --upgrade pip

#Installation of packages required for build
pip install numpy ninja pyyaml mkl mkl-include cmake cffi wheel

#Get source
#Unlike the procedure on the original site, the submodule will be acquired later.
git clone https://github.com/pytorch/pytorch
cd pytorch

# 2020/03/01 13:Switch to the state at the time of 24 (latest state tried this time)
git checkout ace2b4f

#Get submodule
git submodule sync
git submodule update --init --recursive

#Build (about 18 minutes with Threadripper 3960X)
python setup.py install

#Create wheel package (put wheel in pip)
# G:\work\build_pt\pytorch\"Torch" in dist-1.5.0a0+ace2b4f-cp38-cp38-win_amd64.whl "is created
python setup.py bdist_wheel

# ----Build torchvision from here----------------------------------

#Parent folder (g:\work\build_Move to pt)
cd ..

#Introducing additional packages needed to build torchvision
pip install six pillow

#Check if CUDA is used in torchvision
#If this result is True, CUDA will be used (you can also build to force it, see the torchvision page for details).
python -c "import torch; print(torch.cuda.is_available())"

#Get the source code of torchvision
git clone https://github.com/pytorch/vision.git
cd vision

git checkout b2e9565

#Build (a few minutes)
python setup.py install

#Creating a wheel package
python setup.py bdist_wheel

This will create a PyTorch and torchvision wheel file (* .whl) in the dist folder.

Introduction to the usage environment

This is the procedure to install the built PyTorch and torchvision wheel packages in separate virtual environments. Since the build folder uses a lot of disk space, it is convenient to separate the environment that actually uses PyTorch etc. from the virtual environment for build. Here, it is assumed that a virtual environment called `ml``` is created under c: \ venvs \ ``. You can run it from a normal command prompt (not the Visual Studio console)

#Virtual environment(ml)Create
python -m venv c:\venvs\ml
c:\venvs\ml\Scripts\activate.bat
cd /d c:\venvs\ml

#Updates such as packages
python -m pip install --upgrade pip

#Introducing PyTorch and torchvision (including dependent packages)
#For the whl file of PyTorch and torchvision, specify the file created above.
pip install numpy mkl six pillow
pip install "torch-1.5.0a0+ace2b4f-cp38-cp38-win_amd64.whl"
pip install "torchvision-0.6.0a0+b2e9565-cp38-cp38-win_amd64.whl"

This completes the PyTorch 1.5.0a0 environment creation for CUDA 10.2.