While reading "Deep Learning from scratch" (written by Yasuki Saito, published by O'Reilly Japan), I will make a note of the sites I referred to. Part 12 ← → Part 14
I've been learning by running Jupyter Lab on my computer, but memory errors started to occur frequently and I couldn't proceed.
so
I'm going to try Google's Colaboratory.
Colaboratory
There doesn't seem to be an English word for Colaboratory. That's the name Google gave to its service, Collaboration collaborate + laboratory laboratory Is it a coined word from? If it is co + laboratory, is it okay to read it in collaboration? Emphasize the meaning of collaboration If you think of collabora (te) + (labora) story, you can read it as collaboration, but l will increase by one.
Information about colaboratory on Qiita
So, I will go little by little.
So far, I've tried various things, and when I run a program, I refer to many libraries, input files, and pkl files. So, in order to create and reference them on google drive, you have to first understand the structure of folders and files and how to specify them.
For the time being, it seems that there is no problem if you mount the drive and then pass the path to the folder where the libraries and files are located.
#Drive mount
from google.colab import drive
drive.mount('/content/drive')
It seems that you can name the drive part of / content / drive yourself. Does this command give you a name for using Google Drive in your scripts? So, under this drive, there is a folder My Drive as your own area, and under that you can create a folder for script execution, Notebook (~ .ipynb), etc.
This means that all resources available in Colaboratory are under / content / drivename / My Drive. However, since the current directory is / content, you need to be careful when specifying a file with a relative path. Must be'drive name / My Drive / filename'. Of course, you can also change the current drive with the os.chdir () instruction. Changing the current drive only works within that notebook and doesn't seem to affect other notebooks.
I created a folder Colab Notebooks to separate it from other files, and deep_learning under it as a folder to put the example programs of this book. I also created common and dataset folders that I often use in the programs in this book, and placed the downloaded programs and data in them.
Then, pass the path so that the programs under this folder can be referenced from anywhere.
#Add path
import sys
sys.path.append('/content/drive/My Drive/Colab Notebooks/deep_learning/common')
sys.path.append('/content/drive/My Drive/Colab Notebooks/deep_learning/dataset')
The program that refers to these is modified as follows.
#I used to do this when specifying modules in the folder dataset in JupyterLab
from dataset.mnist import load_mnist
#This is fine if you pass through the pass
from mnist import load_mnist
There are many programs that reference the common library, so it seems quite difficult to fix this.
import os
print(os.getcwd()) #Current directory
print(os.pardir) #Parent directory
print(os.path.dirname(os.path.abspath(__file__))) #Should return the directory where this script file is located, but in interactive mode it gives an error
/content .. --------------------------------------------------------------------------- NameError Traceback (most recent call last)
in () 2 print (os.getcwd ()) #current directory 3 print(os.pardir) ----> 4 print(os.path.dirname(os.path.abspath(_file_))) 5 NameError: name '_file_' is not defined
The interactive mode of Google Colaboratory seems to be running in the current directory / content. Running the above script in a different folder will have the same result, the current directory will be / content. Also, it seems that \ __file \ __ cannot be used in interactive mode. Since the current directory is always / content, it doesn't make sense to use it.
However, it seems that \ __file \ __ can be used in the script saved with a file name. The following script works fine and loads a file in the same folder as the script.
# coding: utf-8
import sys, os
sys.path.append(os.pardir) #Settings for importing files in the parent directory
import numpy as np
from mnist import load_mnist
import matplotlib.pyplot as plt
def showImg(x):
example = x.reshape((28, 28))
plt.figure()
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(example, cmap=plt.cm.binary)
plt.show()
return
(x_train, t_train), (x_test, t_test) = load_mnist(flatten=True, normalize=False)
img = x_train[1]
label = t_train[1]
print(img.shape) # (784,)
img = img.reshape(28, 28) #Transform the shape to the original image size
print(img.shape) # (28, 28)
print(label) # 0
showImg(img)
(784,) (28, 28) 0
In the imported mnist.py, the file is specified like this
mnist.Script setting mnist data directory with py
dataset_dir = os.path.dirname(os.path.abspath(__file__))
save_file = dataset_dir + "/mnist.pkl"
You can read mnist.pkl from the folder'/ content / drive / My Drive / Colab Notebooks / deep_learning / dataset' where mnist.py is located.
# coding: utf-8
import matplotlib.pyplot as plt
from matplotlib.image import imread
img = imread('/content/drive/My Drive/Colab Notebooks/deep_learning/dataset/lena.png') #Loading images
plt.imshow(img)
plt.show()
It seems to work somehow, so after actually running the program in the book and checking it, I'm going to try to identify the dog and cat photos that were too heavy to move.
Part 12 ← → Part 14 Click here for the table of contents of the memo Unreadable Glossary
Import .py files on Google Drive with Google Colaboratory Find out why you can't get the directory name of a running script using file Welcome to Colaboratory Deep Learning Practice Tips on Colaboratory
Recommended Posts