[PYTHON] I tried to implement Cifar10 with SONY Deep Learning library NNabla [Nippon Hurray]

It seems that SONY did it

Sony accelerates the evolution of AI. Open source deep learning core library

• Quoted from gizmodo

SONY has released its own Deep Learning library as open source. Official site Click here if you are interested

Isn't Japan doing its best, which is ridiculed as an IT underdeveloped country in some areas? In particular, major Japanese companies do not hear good rumors about software, so ...

Recently, such a blog about SONY has become a hot topic ... Retired from Sony Corporation

It's a big company, so I don't know if everything is as per this blog, I'm sure I didn't have a good impression. Personally, I don't really feel like supporting it because it's a Japanese company. Since it's a big deal, let's write an impression of using the library on Qiita!

Yes, everyone loves Cifar10

Click here for code

For the time being, I did it by defeating it. Yeah. Don't worry about the poor quality. ~~ I wanted to write an article about using NNabla first on Qiita, so this is the result of finishing it at super speed! !! !! ~~ Don't worry about the accuracy as you don't have time and the model is super suitable. .. I just wanted to touch the API. ..

main.py

def convolution(x):
  x = x.reshape([BATCH_SIZE, IMAGE_DEPTH, IMAGE_HEIGHT, IMAGE_WIDTH])
  with nn.parameter_scope("conv1"):
    output = PF.convolution(x, 16, (5, 5), stride=(2, 2), pad=(1, 1))
    output = F.relu(output)
  
  with nn.parameter_scope("conv2"):
    output = PF.convolution(output, 32, (3, 3), stride=(1, 1), pad=(1, 1))
    output = F.relu(output)
  
  with nn.parameter_scope("conv3"):
    output = PF.convolution(output, 64, (3, 3), stride=(1, 1), pad=(1, 1))
    output = F.relu(output)

  output = output.reshape([BATCH_SIZE, int(output.size / BATCH_SIZE)])

  with nn.parameter_scope("fc1"):
    output = PF.affine(output, 1024)
    output = F.relu(output)
  
  with nn.parameter_scope("fc2"):
    output = PF.affine(output, 256)
    output = F.relu(output)
  
  with nn.parameter_scope("softmax"):
    output = PF.affine(output, 10)
    output = F.softmax(output)
  
  return output

It's a CNN like this. The API reference Parametric Functions is an abstract API that makes it fairly easy to define a network. The low-layer one is this. (I haven't looked at it in detail ...)

Step: 00290 Train loss: 7.17657 Train accuracy: 0.34375
Step: 00300 Test loss: 7.22971 Test accuracy: 0.34990
Step: 00300 Train loss: 7.23585 Train accuracy: 0.28125
Step: 00310 Train loss: 7.26531 Train accuracy: 0.28125
Step: 00320 Train loss: 7.15616 Train accuracy: 0.37500
Step: 00330 Train loss: 7.19948 Train accuracy: 0.29688
Step: 00340 Train loss: 7.23404 Train accuracy: 0.26562
Step: 00350 Train loss: 7.13390 Train accuracy: 0.42188
Step: 00360 Train loss: 7.27805 Train accuracy: 0.20312
Step: 00370 Train loss: 7.08152 Train accuracy: 0.46875
Step: 00380 Train loss: 7.17094 Train accuracy: 0.34375
Step: 00390 Train loss: 7.12861 Train accuracy: 0.39062
Step: 00400 Test loss: 7.21064 Test accuracy: 0.36996

The execution result is output to the console like this. It feels lighter than writing a similar network in TensorFlow.

Difficulties

I usually only touch TensorFlow, and in fact I have never used Deep Learning libraries other than TF. .. It's embarrassing. I've used scratches to deepen my understanding.

To be honest, I don't have much difficulty. If you have used other libraries, can you use them without any discomfort? It's easy because you can define abstract layers. I can't say anything because I haven't used other libraries, but I could use this kind of code with the same feeling.

Speaking of the difficult points, it's been a long time since I used python2 system. I completely forgot the int division specification.

main.py

def accuracy(p, t):
  pred_and_label = [(np.argmax(_p), np.argmax(_t)) for _p, _t in zip(p.d, t.d)]
  return float(len(filter(lambda x: x[0] == x[1], pred_and_label))) / float(len(p.d))

I am calculating the correct answer rate of the output result here. This guy stayed at 0 all the time, and I was thinking for a while that it was strange. .. This has been fixed to float, so it will be output properly.

~~ Hmm. I mostly use python3 for business, so I want to support 3 systems as soon as possible here. ~~ It seems that it supports python3! Thank you for your comment! (2017/08/18)

** Addition **

Maybe I had a hard time, or maybe I just couldn't find it because I didn't have time, I couldn't find anything that corresponds to None in the shape definition of TensorFlow. The one that defines a shape like [?, 3, 64, 64]. Looking at the code on Github of Sample, it seems that the test data was calculated by repeating batches of the same size as the training, so I did that too. I'm not honest that the batch size can only be fixed, so I wonder if there is any way. .. Let's examine that area later. ~~ If you have time! !! ~~

Summary

~~ Personally, use TensorFlow ~~ I want the Deep Learning area to get excited, and I want you to do your best. If I have a free time, I think I should touch it more and pull it.

Also, I'm a little worried about this DynamicNN, so I'll touch it again when I have time. It sounds like a multi-paradigm of Define by run and Define and run.

Please let me know if there is something wrong! !!