[PYTHON] Implementation of a convolutional neural network using only Numpy

Introduction

To understand the convolutional neural network, I implemented the algorithm in Python using only Numpy, without relying on the library for deep learning. In the future, I would like to improve it to be compatible with GPU. In addition, since it has not been verified whether the inverse error is properly propagated, the one under development is posted.

GitHub https://github.com/NaotoMasuzawa/Deep_Learning/tree/master/Python_CNN

Referenced code and technical book

In implementing the code, I referred to Yusuke Sugomori's GitHub and technical books. I am very grateful.

GitHub https://github.com/yusugomori Technical book https://www.amazon.co.jp/Java-Learning-Essentials-Yusuke-Sugomori-ebook/dp/B01956B5RQ

Outline

In addition to the normal convolutional neural network, a dropout that stochastically reduces some of the neurons to 0 is implemented in the hidden layer. In addition, ReLU is used as the activation function.

Code structure and annotations

Build.py Set the layer configuration and parameters here. Also, when executing the code, You should be able to set the parameters here and run it with python Build.py.

Convolutional_Neural_Network.py Each class rises here depending on the structure of the layer.

Conv_Pool_Layer.py A convolution layer and a pooling layer are implemented. For statements are often used for the tatami layer, Since the relationship between the indexes becomes complicated, I made each one meaningful. Therefore, the width of a part of the code is longer.

Hidden_Layer.py A dropout on a regular multi-layer perceptron used in fully connected layers It is implemented.

Logistic_Regression.py Logistic regression is implemented.

utils.py The activation function (ReLU) is implemented. If you want to compare it with other activation functions (such as sigmoid function), write it here.

Finally

Since this code is not used to perform any heavy processing, it may contain bugs. In that case, I would appreciate it if you could report it.

Recommended Posts

Implementation of a convolutional neural network using only Numpy
Neural network implementation (NumPy only)
Implementation of "blurred" neural network using Chainer
Simple neural network implementation using Chainer
What is a Convolutional Neural Network?
Rank learning using neural network (Implementation of RankNet by Chainer)
Implementation of 3-layer neural network (no learning)
Simple neural network implementation using Chainer-Data preparation-
Simple neural network implementation using Chainer-Model description-
Understand the number of input / output parameters of a convolutional neural network
Build a classifier with a handwriting recognition rate of 99.2% with a TensorFlow convolutional neural network
I made an image discrimination (cifar10) model using a convolutional neural network.
Simple neural network implementation using Chainer-optimization algorithm setting-
Reinforcement learning 10 Try using a trained neural network.
Another style conversion method using Convolutional Neural Network
Visualize the inner layer of a neural network
Implement Convolutional Neural Network
Convolutional neural network experience
[Python] Implementation of clustering using a mixed Gaussian model
Model using convolutional neural network in natural language processing
The story of making a music generation neural network
Bayesian optimization implementation of neural network hyperparameters (Chainer + GPyOpt)
Basics of PyTorch (2) -How to make a neural network-
Implement a 3-layer neural network
Implementation of TF-IDF using gensim
Neural network implementation in python
A memorandum of using eigen3
Implementation of VGG16 using Keras created without using a trained model
Try building a neural network in Python without using a library
Construction of a neural network that reproduces XOR by Z3
CNN Acceleration Series ~ FCNN: Introduction of Fourier Convolutional Neural Network ~
How to easily draw the structure of a neural network on Google Colaboratory using "convnet-drawer"
I tried a convolutional neural network (CNN) with a tutorial on TensorFlow on Cloud9-Classification of handwritten images-
Implementation of a simple particle filter
PRML Chapter 5 Neural Network Python Implementation
I implemented a two-layer neural network
How to save only a part of a long video using OpenCV
Simple neural network theory and implementation
Implementation of desktop notifications using Python
Touch the object of the neural network
Getting a combination of elements using itertools
Survivor prediction using kaggle's titanic neural network [80.8%]
Compose with a neural network! Run Magenta
A memorandum of using Python's input function
Implementation of dialogue system using Chainer [seq2seq]
Impressions of using Flask for a month
[Python] Get a list of folders only
[Chainer] Document classification by convolutional neural network
Python vs Ruby "Deep Learning from scratch" Chapter 3 Implementation of 3-layer neural network
Save an array of numpy to a wav file using the wave module
[Deep Learning from scratch] Initial value of neural network weight using sigmoid function
Extract elements (using a list of indexes) in a NumPy style from a Python list / tuple
Experiment with various optimization algorithms with a neural network
Verification of Batch Normalization with multi-layer neural network
Recognition of handwritten numbers by multi-layer neural network
[Deep Learning from scratch] Initial value of neural network weight when using Relu function