[PYTHON] Simple neural network implementation using Chainer

Until the last time, I wrote about the basic objects that make up Chainer. From this time, I would like to actually implement a neural network using those objects.

Overall view of neural network

When writing a program that composes a neural network, the overall view is roughly composed of the following five.

  1. Data preparation

It is a program that prepares data for training the model. There are various types of data such as sentences, images, and voices, but we process them into the form of data suitable for the neural network to be constructed.

  1. Model description

Describe a class that inherits the Chain class. For details, refer to here.

class MyChain(Chain):
    def __init__(self):
        super(MyChain, self).__init__(
Declaration of a function containing parameters
        )
    def __call__(self, x, y):
Loss (error) function
  1. Optimization algorithm settings (SDG, Adam, etc.)

Describes the process of optimizing the model (minimizing the error). For more information, see here

model = MyChain()
optimizer = optimizer.Adam()
optimizer.setup(model)
  1. Learning

Learning is performed a specified number of times. Depending on the item, early termination may be performed, but this time we will simply specify epoch (number of learnings). It seems that the code is almost promised from gradient initialization to parameter update.

for epoch in range(Number of repetitions):
    model.zerograds()           #Gradient initialization
    loss = model(train, target) #Error calculation
    loss.backward()             #Gradient calculation
    optimizer.update()          #Parameter update
  1. Result output

Save the model of the training result and perform the test.

Direction from the next time

I don't want to stuff this information into one article, so from the next time

  1. Data preparation
  2. Model description
  3. Optimization algorithm settings
  4. Learning
  5. Result output

I will write about it in small parts. Up to here for this time

reference

Takayoshi Yamashita Deep learning Kodansha that can be seen in the illustration Hiroyuki Shinno Practical deep learning with Chainer-How to implement complex NN-Ohmsha

Recommended Posts

Simple neural network implementation using Chainer
Implementation of "blurred" neural network using Chainer
Simple neural network implementation using Chainer-Data preparation-
Simple neural network implementation using Chainer-Model description-
Simple neural network theory and implementation
Rank learning using neural network (Implementation of RankNet by Chainer)
Neural network starting with Chainer
Neural network implementation in python
Neural network implementation (NumPy only)
Bayesian optimization implementation of neural network hyperparameters (Chainer + GPyOpt)
Implementation of a convolutional neural network using only Numpy
Implementation of a two-layer neural network 2
PRML Chapter 5 Neural Network Python Implementation
Simple classification model with neural network
Survivor prediction using kaggle's titanic neural network [80.8%]
Implementation of 3-layer neural network (no learning)
Implementation of dialogue system using Chainer [seq2seq]
Try using TensorFlow-Part 2-Convolutional Neural Network (MNIST)
[Chainer] Document classification by convolutional neural network
Parametric Neural Network
Reinforcement learning 10 Try using a trained neural network.
Another style conversion method using Convolutional Neural Network
Author estimation using neural network and Doc2Vec (Aozora Bunko)
Overview of DNC (Differentiable Neural Computers) + Implementation by Chainer
Implement Convolutional Neural Network
Model using convolutional neural network in natural language processing
Implement Neural Network from 1
Implement feedforward neural network in Chainer to classify documents
Convolutional neural network experience
Implementation of Chainer series learning using variable length mini-batch
Precautions when using Chainer
Try building a neural network in Python without using a library
Implement a 3-layer neural network
Implementation of TF-IDF using gensim
Neural network with Python (scikit-learn)
3. Normal distribution with neural network!
# 1 [python3] Simple calculation using variables
Pytorch Neural Network (CNN) Tutorial 1.3.1.
4. Circle parameters with neural network!
TensorFlow Tutorial-Convolutional Neural Network (Translation)