I tried Deep Learning using Pylearn2. I have not tested it with decent data, and there is no decent output, so it would be helpful if you could read it for the purpose of grasping the atmosphere of Pylearn 2 such as "Hmm, Pylearn 2 looks like this" I will.
Please refer to the official document for the time being. Depending on the version of theano, an error will occur. In that case, please refer to stackoverflow below.
For the time being, if you read the Computer Vision Cutting-edge Guide 6 below, read the Deep Learning Tutorials, and implement it with theano, you will understand Deep Learning somehow. By the way, I can only understand it somehow. .. Regarding the implementation in theano, there is a post with an easy-to-understand explanation of the method in sinhrks' blog, so I think that it will reduce the learning cost. Also, Introduction to Theano by Intelligent Information Systems Co., Ltd. will be very helpful. We recommend that you read it together.
Commented out is intentionally left for trial and error. I'm sorry it's hard to see, but please understand.
Create and save toy_train.csv and toy_train.pkl (saved as Pylearn2 serial object).
create_toy_dataset.py
import numpy
from pylearn2.datasets import DenseDesignMatrix
from pylearn2.utils import serial
class ToyDataset(DenseDesignMatrix):
def __init__(self):
rng = numpy.random.RandomState(seed=42)
data = numpy.array( [rng.randint(0,2,200) for _ in range(1000)] )
self.data = data
super(ToyDataset, self).__init__(X=data) #, y=self.y)
def main():
train = ToyDataset()
train.use_design_loc('./toy_train_design.npy')
rng = numpy.random.RandomState(seed=42)
train_csv_path = './toy_train.csv'
train_pkl_path = './toy_train.pkl'
data = numpy.array( [rng.randint(0,2,200) for _ in range(1000)] ,dtype=numpy.int)
numpy.savetxt(train_csv_path, data, delimiter=',',fmt='%d')
serial.save(train_pkl_path, train)
if __name__ == '__main__':
main()
I tried to read the csv file directly and move it, but it didn't work, so I read the pkl file and use it. The CSV Dataset delimiter is a comma by default. If delimiter is not specified in numpy.savetxt, please rewrite delimiter to half-width space in CSVDataset of yaml file. If there is no label, write expect_labels: False. If there is no header, write expect_headers: False. For details, please refer to the Official Document. For the data structure when creating your own data, [Pylearn2 I want to get started](http://laughing.hatenablog.com/entry/2013/11/30/pylearn2_%E5%85%A5%E9%96% 80% E3% 81% 97% E3% 81% 9F% E3% 81% 84% E7% B7% A8) will be helpful. As you can see, the coefficients are set appropriately, so please do not refer to them. For the practice of Deep Learning, I summarized A Practical Guide to Training Redstricted Boltzmann Machine Deep Learning from RBM-Black With magic ~ will be helpful. Also, I personally found the momentum Implementation of Dropout and regularization of weights helpful.
grbm_dbm.yaml
!obj:pylearn2.train.Train {
dataset: &data !obj:pylearn2.datasets.binarizer.Binarizer {
raw: &raw_train !pkl: "toy_train.pkl",
},
#dataset: &train !obj:pylearn2.datasets.csv_dataset.CSVDataset {
# path: %(path)s,
# start: %(start)i,
# stop: %(stop)i,
# expect_labels: False,
# expect_headers: False
#},
model: !obj:pylearn2.models.dbm.DBM {
batch_size: %(batch_size)i,
niter: 5,
visible_layer: !obj:pylearn2.models.dbm.BinaryVector {
nvis: %(nvis)i,
bias_from_marginals: *raw_train,
},
hidden_layers: [
!obj:pylearn2.models.dbm.BinaryVectorMaxPool {
layer_name: 'h1',
detector_layer_dim: %(detector_layer_dim1)i,
pool_size: 1,
irange: .05,
init_bias: -2.,
},
!obj:pylearn2.models.dbm.BinaryVectorMaxPool {
layer_name: 'h2',
detector_layer_dim: %(detector_layer_dim2)i,
pool_size: 1,
irange: .05,
init_bias: -2.,
},
!obj:pylearn2.models.dbm.BinaryVectorMaxPool {
layer_name: 'y',
detector_layer_dim: %(detector_layer_dim3)i,
pool_size: 1,
irange: .05,
init_bias: -2.,
}
#!obj:pylearn2.models.dbm.Softmax {
# layer_name: 'y',
# max_col_norm: 1.9365,
# #supervised: False,
# n_classes: 10,
# irange: .05,
#}
],
},
algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
learning_rate: .05,
learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum {
init_momentum: .5,
},
monitoring_batches: %(monitoring_batches)i,
monitoring_dataset: { 'train': *data
#'test': !obj:pylearn2.datasets.csv_dataset.CSVDataset {
# path: %(path)s,
# start: %(test_start)i,
# stop: %(test_stop)i,
# expect_labels: False,
# expect_headers: False
#},
},
cost: !obj:pylearn2.costs.cost.SumOfCosts {
costs: [
!obj:pylearn2.costs.dbm.VariationalPCD {
num_chains: 100,
num_gibbs_steps: 5
},
!obj:pylearn2.costs.dbm.WeightDecay {
coeffs: [.0001, .0001, .0001]
},
!obj:pylearn2.costs.dbm.TorontoSparsity {
targets: [.2, .2, .2],
coeffs: [.001, .001, .001],
}
],
},
termination_criterion: !obj:pylearn2.termination_criteria.EpochCounter { max_epochs: %(max_epochs)i },
update_callbacks: [
!obj:pylearn2.training_algorithms.sgd.ExponentialDecay {
decay_factor: 1.000015,
min_lr: .000001
}
]
},
extensions: [
!obj:pylearn2.training_algorithms.learning_rule.MomentumAdjustor {
final_momentum: .9,
start: 5,
saturate: 6
}
],
save_path: "%(save_path)s/grbm_dbm.pkl",
save_freq: 1
}
training I get an error if I don't put ToyDataset class in the dataset of pylearn2, but it seems to work without problems even if I write it directly in the training source, so there is a description in the code below (meaningless).
train_dbm.py
import os
from nose.plugins.skip import SkipTest
from theano import config
import pylearn2
from pylearn2.datasets import DenseDesignMatrix
from pylearn2.testing import skip
from pylearn2.testing import no_debug_mode
from pylearn2.utils.serial import load_train_file
from pylearn2.config import yaml_parse
class ToyDataset(DenseDesignMatrix):
def __init__(self):
rng = numpy.random.RandomState(seed=42)
data = numpy.array( [rng.randint(0,200,200) for _ in range(1000)] )
super(ToyDataset, self).__init__(X=data)#, y=self.y)
@no_debug_mode
def train_yaml(yaml_file):
train = yaml_parse.load(yaml_file)
train.main_loop()
def train(yaml_file_path, save_path):
yaml = open("{0}/grbm_dbm.yaml".format(yaml_file_path), 'r').read()
hyper_params = {'batch_size': 100,
'nvis': 200,
'detector_layer_dim1': 300,
'detector_layer_dim2': 300,
'detector_layer_dim3': 10,
'monitoring_batches': 10,
'max_epochs': 100,
'save_path': save_path}
yaml = yaml % (hyper_params)
train_yaml(yaml)
def train_dbm():
skip.skip_if_no_data()
yaml_file_path = '.'
save_path = '.'
train(yaml_file_path, save_path)
if __name__ == '__main__':
train_dbm()
dbm_result.py
import numpy as np
import pickle
import sys
import theano
from pylearn2.space import VectorSpace
from pylearn2.datasets import DenseDesignMatrix
class ToyDataset(DenseDesignMatrix):
def __init__(self):
rng = numpy.random.RandomState(seed=42)
data = numpy.array( [rng.randint(0,2,200) for _ in range(1000)] )
super(ToyDataset, self).__init__(X=data)#, y=self.y)
def simulate(inputs, model):
print dir(model)
space = VectorSpace(inputs.shape[1])
X = space.get_theano_batch()
q = model.mf(space.format_as(X, model.get_input_space()))
f = theano.function([X], q[-1])
result = []
batch_size = 100
for x in xrange(0, len(inputs), batch_size):
result.extend(f(inputs[x:x + batch_size]))
print "len:", len(result)
print result[0]
return result
def countCorrectResults(outputs):
correct = 0;
for output in outputs:
for out_vec in output:
print np.argmax(out_vec)
def score(dataset, model):
outputs = simulate(dataset.X, model)
countCorrectResults(outputs)
model = pickle.load(open('grbm_dbm.pkl')) #Post-learning model
test_data = pickle.load(open('toy_train.pkl')) #Data used for learning
score(test_data, model)
As a result of moving the above code, I got an unclear result that the probability value becomes max at index 6 for all data w I feel that something is wrong, or something is fundamentally wrong. ..
There are some modules that are on github but not mentioned in the Pylearn2 documentation, so I think it's quite difficult to work properly. I don't think Pylearn2 is catching up with the documentation, but there are some that describe the outline of the method, so take a quick look at the document to see if there is something that seems to be efficient with a method that you do not know. , I feel that I can learn efficiently if I read the paper of the method I was interested in. (I just feel ...)
In addition to what I did above, I was doing stupid things and tried to see if I could run unsupervised RBM with MLP. However, it seems that MLP only accepts supervised learning (an error occurs during monite ring).
We apologize for the inconvenience, but we would appreciate it if you could point out any mistakes.
Recommended Posts